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1.0  EXECUTIVE  SUMMARY 


A  study  has  been  conducted  in  1995  to  evaluate  the  training  value  of  the  U.S.  Marine  Corps 
Team  Tactical  Engagement  Simulator  (TTES),  as  perceived  by  28  subject  matter  experts  from  six 
military  and  civilian  agencies.  This  study  was  a  follow-on  to  a  similar  1994  evaluation.  TTES  is 
being  developed  to  provide  a  deployable,  affordable,  realistic  simulated  combat  environment  where 
military  and  civilian  personnel  can  receive  training  in  making  discretionary  decisions  (learning 
when  to  engage  or  not  engage  a  potential  adversary)  as  well  as  marksmanship. 

All  of  the  evaluators  were  experienced  small  arms  trainers;  23  were  military  officers  or 
enlisted  personnel  with  infantry  or  security  backgrounds  and  five  were  civilian  law  enforcement 
officers.  The  study  participants  used  the  current  TTES  system  or  observed  it  in  use  and  received  a 
briefing  on  the  capabilities  envisioned  for  the  ultimate  system.  Then  each  completed  a  detailed 
questionnaire,  using  a  5-point  scale  to  rate  the  Potential  Usefulness  of  incorporating  numerous 
factors  into  TTES  simulations.  Respondents  also  provided  their  opinions  on  TTES  Current 
Capability  for  meeting  training  needs  and  on  required  Realism  Levels. 

Respondents  generally  gave  very  high  calculated  ratings  (86  to  98%,  with  an  average  of 
94.6%)  for  the  usefulness  of  including  the  listed  sets  of  factors  in  TTES  —  substantially  higher 
ratings  than  were  given  by  participants  in  the  1994  survey  (an  average  of  83.8%).  From  among 
the  17  categories  included  in  the  survey,  respondents  indicated  that  Sounds,  General  Objects,  and 
Simulated  Humans  were  the  most  useful  to  simulate,  though  distances  between  all  adjacent  ratings 
were  small  and  even  the  bottom-rated  item.  Eye  Movements  for  Adversaries,  was  rated  at  86.7%. 
Good  to  very  good  realism  is  needed  for  all  objects,  situations,  and  events. 

As  was  die  case  in  1994,  the  1995  ratings  emphasized  how  important  it  is  that  TTES  not 
constrain  the  activities  of  its  users  any  more  than  absolutely  necessary.  The  provided  equipment 
should  allow  users  to  carry  out  common  battlefield  activities  and  procedures  during  training  in  a 
natural,  normal  manner  so  the  skills  gained  will  transfer  directly  to  combat  situations. 

Modeling  simulated  adversaries  in  reasonably  realistic  detail  was  considered  more  important 
by  the  1995  respondents  than  by  those  in  1994.  These  results  alleviate  1994  concerns  that  there 
might  be  a  general  attitude  that  the  adversary  is  inherently  less  capable  and  thus  requires  a  less 
sophisticated  representation  in  TTES.  Such  a  perception  would  be  extremely  dangerous  in  real- 
world  combat,  and  nothing  in  training  must  be  allowed  to  reinforce  this  misconception  in  any  way. 
The  adversaries  engaged  during  TTES  training  must  appear  to  be  bright,  clever,  and  fully  capable 
of  carrying  out  well-planned  and  skilled  coordinated  tactical  movements.  Neutrals  and  friendlies 
must  be  realistic  enough  so  intelligent  judgments  are  needed  when  deciding  whether  to  engage. 

TTES  Current  Capability  was  judged  to  have  improved  considerably  in  the  past  year.  1 995 ’s 
60.2%  average  rating  was  up  significantly  from  1994’s  42.0%.  Modeling  of  the  Environments 
category  (urban,  jungle,  desert,  etc.)  was  considered  the  best  at  present,  while  the  Environmental 
Effects  category  (weapon  impacts,  reduced  visibility,  etc.)  was  rated  among  the  lowest. 

The  1994  survey  respondents’  ratings  indicated  that  the  envisioned  TTES  system  potentially 
might  meet  between  80  and  85%  of  their  perceived  needs  for  marksmanship  and  discretionary 
decision  training.  In  1995  the  calculated  overall  average  Potential  Usefulness  rating  was  over 
94%.  These  findings  suggest  that  a  simulator  system  such  as  TTES  eventually  could  result  in 
phenomenal  savings  in  time,  ammunition,  and  other  costs  related  to  training,  while  providing  the 
capability  to  train  to  higher  standards  and  to  challenge  trainees  in  ways  not  currently  possible. 
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2.0  INTRODUCTION 


2.1  BACKGROUND 

The  Team  Tactical  Engagement  Simulator  (TTES)  project  is  part  of  a  service-wide  effort  to 
utilize  advanced  modeling  and  simulation  techniques  for  training  individual  combatants.  These 
individual  combatants  include  infantry,  security  forces,  special  operations  forces,  and  law 
enforcement  personnel.  The  TTES  research  and  development  effort  is  sponsored  by  the  U.S. 
Marine  Corps  and  is  being  carried  out  under  the  leadership  of  the  Naval  Air  Warfare  Center 
Training  Systems  Division  (NAWC-TSD),  Orlando,  Florida.  Primary  developer  of  the  simulation 
software  is  the  Institute  for  Simulation  and  Training  (1ST),  University  of  Central  Florida,  Orlando, 
Florida. 

The  TTES  Advanced  Technology  Demonstration  effort  was  begun  in  fiscal  year  1993  to 
develop  core  technology  for  individual  combatant  and  small-unit  force-on-force  training  devices, 
and  will  continue  through  fiscal  year  1996.  Two  Defense  Modeling  and  Simulation  Office  awards 
and  associated  teaming  with  other  Department  of  Defense  agencies  have  enhanced  TTES 
contributions  to  joint  Individual  Combat  Simulation  System  efforts.1 

It  is  anticipated  that  the  TTES  training  systems  will  be  used  by  the  U.S.  Marine  Corps,  U.S. 
Army,  U.S.  Air  Force,  Security  Police,  Special  Operations  Forces,  and  various  civilian  law 
enforcement  agencies  such  as  the  Federal  Bureau  of  Investigation  and  the  U.S.  Drug  Enforcement 
Agency.  Thus  it  is  important  to  ensure  that  the  planned  TTES  simulations  have  the  capability  to 
meet  the  needs  of  both  trainers  and  trainees  in  those  various  agencies. 

TTES  emphasizes  tactical  decision-making  and  close  combat  marksmanship  skills  in  a 
Dynamic  Synthetic  Environment  (DSE),  complementing  live-fire  and  field  training.  The  DSE 
includes  (1)  a  dynamic  representation  of  the  physical  environment  and  (2)  behavioral 
representation  of  friendlies,  hostiles,  and  neutrals.  That  is,  training  can  be  conducted  in  a  realistic 
simulated  combat  environment  where  trainees  will  encounter  computer-controlled  hostile  and 
neutral  (CCH/N)  entities  whose  high-fidelity  simulated  combat  behavior  will  closely  emulate  that 
of  actual  hostile  and  neutral  units  and  individuals.  An  initial  attempt  at  defining  battlefield 
behaviors  of  five  kinds  of  hostile  and  neutral  individuals  has  been  completed  for  TTES  modeling  of 
CCH/Ns  as  a  companion  effort  to  this  determination  of  perceived  TTES  training  value.2, 3 

TTES  is  not  merely  a  marksmanship  trainer,  although  it  will  provide  both  individuals  and 
teams  the  opportunity  to  practice  small  arms  and  other  weapons  skills.  More  important,  TTES 
will  train  when  to  shoot  as  well  as  how  to  shoot.  TTES  eventually  will  be  capable  of  emulating  a 


1  Marine  Corps  System  Command.  Team  Tactical  Engagement  Simulator  (TTES)  Advanced 
Technology  Demonstration,  by  F.J.  Wysocki  and  D.H.  Fowlkes.  Quantico,  Vir.,  Amphibious  Warfare 
Technology  Directorate,  February  1994. 

2  Naval  Postgraduate  School.  Battlefield  Behaviors  of  Neutrals  and  Hostiles:  Models  for  the  Team 
Tactical  Engagement  Simulator  (TTES),  by  J.H.  Lind.  Monterey,  Calif.,  NPS,  September  1995  (NPSOR- 
95-006) 

3  Naval  Air  Warfare  Center  Weapons  Division.  Behavior  Representation  for  the  Team  Tactical 
Engagement  Simulator  (TTES),  by  J.H.  Lind.  China  Lake,  Calif.,  NAWCWPNS,  in  preparation. 
(NAWCWPNS  TP  8224) 
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variety  of  humans,  ranging  from  desperate  criminals  and  mobs,  through  well-trained  or  marginally- 
trained  enemy  forces,  to  neutral  or  unfriendly  villagers,  friendly  allies,  hostages,  and  prisoners. 
Using  TTES,  trainees  will  have  the  opportunity  to  hone  their  skills  related  to  recognizing  the  intent 
and  level  of  hostility  of  individuals  and  groups,  and  learn  to  make  intelligent  decisions  concerning 
when  to  engage  and  when  to  withhold  fire.  Indeed,  it  is  anticipated  that  improved  tactical  and 
decision  making  skills  will  be  the  most  significant  TTES  training  payoff. 

2.1.1  Envisioned  TTES  System 

TTES  is  a  virtual  reality  training  device  under  development  in  the  Marine  Corps  technology 
base.  TTES  is  intended  as  a  supplement  and  complement  to  live-fire  and  field  training,  not  as  a 
substitute.  The  software  developed  in  the  effort  can  be  applied  as  a  modular  upgrade  to 
compatible  small  arms  training  systems  for  infantry,  security  force,  and  special  operations  units. 
The  envisioned  TTES  product  is  projected  for  fielding  in  the  2002  to  2006  timeframe. 

The  TTES  training  device  will  be  deployable,  affordable,  and  easy  to  produce.  Oriented  to 
individual  combatants  and  small  units,  TTES  technology  will  contribute  to  operational  readiness 
by  enabling  mission  preparation  in  operationally  relevant  synthetic  environments.  Trainees  also 
will  be  able  to  utilize  a  variety  of  combat  weapons  in  realistic  tactical  situations  while  traversing  a 
simulated  environment  that  includes  open  terrain,  villages,  and  various  kinds  of  buildings.  When 
TTES  is  provided,  an  appropriate  terrain  database  of  an  objective  area  will  be  available,  along 
with  computer-controlled  hostiles  characterized  to  emulate  the  behaviors  of  the  expected  enemy. 
Such  capabilities  will  enable  mission  previews  and  mission  rehearsals  specific  to  the  anticipated 
combat  situation  in  a  realistic  emulation  of  the  objective  area. 

The  envisioned  TTES  system  will  fully  immerse  fire  team  members  in  a  common  virtual 
reality  using  wide-angle  screens  and/or  helmet-mounted  displays  with  audio  capabilities,  for 
training  as  a  coordinated  tactical  unit  in  the  same  synthetic  environment.  Multiple  trainees  will  be 
able  to  interact  with  each  other  while  linked  via  a  radio  frequency  network  using  Distributed 
Interactive  Simulation  (DIS)  protocols,  so  that  they  are  not  constrained  by  signal-carrying  wires. 
Trainees  also  will  be  networked  with  CCH/Ns  via  DIS  protocols,  enabling  emulation  of  force-on- 
force  engagements  against  various  adversaries  in  areas  where  neutral  and  friendly  forces  are 
present. 

The  refinement  of  discretionary  and  tactical  decision  skills  will  be  the  most  significant  TTES 
payoff.  Other  payoffs  include  maintenance  of  perishable  skills,  realistic  training  in  expeditionary 
situations  such  as  during  prolonged  shipboard  deployments,  and  weapon  virtual  prototyping. 

2.1.2  Current  TTES  Emphasis 

The  current  thrust  of  TTES  is  Military  Operations  in  Urban  Terrain  (MOUT).  This  emphasis 
was  chosen  for  mission  relevance  and  for  a  sufficient  challenge  during  technology  evolution. 
Tactical  decision  skills  and  close  combat  marksmanship  skills  are  the  training  facets  emphasized  at 
this  time. 

TTES  development  is  focused  initially  on  general  purpose  infantry.  Scenarios  for  training 
security  forces  are  next  in  line.  Training  for  special  operations  units  and  rear-area  security 
situations  will  follow.  The  basic  technology  will  meet  requirements  for  and  be  applicable  to 
training  programs  for  all  military  services.  Near-term  TTES  capabilities  also  will  be  useful  for 
various  civilian  law  enforcement  agencies. 
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2.1.3  TTES  Equipment 

The  current  TTES  evaluation  hardware  suite  is  illustrated  in  Figure  1 .  Each  TTES  suite 
includes  an  8-  by  10-foot  rear-projection  display  screen,  a  projector,  an  M16-A2  demilitarized  rifle 
and  weapon  monitor,  a  head-position  tracker,  microphone  and  headphones,  a  foot  pedal  for 
movement  control,  and  a  computer  graphics  generator,  along  with  the  TTES  software  that  runs  the 
program.  Two  or  more  suites  can  be  used  concurrently  for  team  training.  The  trainee  aims  the 
weapon  as  desired,  looks  through  the  sight,  and  pulls  the  trigger  to  fire  a  virtual  round  that  follows 
a  ballistic  path  through  the  virtual  battle  space. 

Pressure  on  the  forward  end  of  the  foot  pedal  causes  the  system  to  move  the  trainee  forward 
through  the  environment  at  a  rate  proportional  to  the  amount  of  pressure.  Pressure  on  the  back  of 
the  plate  results  in  the  appearance  of  moving  backwards.  The  head-tracking  device  is  attached  to  a 
helmet  that  is  worn  during  the  training  exercise.  Moving  the  head  left  or  right  causes  the  projected 
scene  to  move  appropriately  to  display  new  areas.  The  microphone  and  headphones  enable 
communications  among  trainees  in  separate  TTES  stations  who  are  practicing  team  exercises. 


The  envisioned  TTES  system  for  2002,  shown  in  Figure  2,  includes  weapon  and  body 
monitors,  non-tethered  tracking  emitters  that  transmit  signals  related  to  human  motion,  a  foot- 
controlled  movement  device,  a  head-mounted  visual  and  audio  display,  a  trainee  computer  pack, 
and  a  master  computer  with  a  radio  frequency  DIS  network.  Eventually  (in  the  2006  timeframe) 
the  use  of  DIS  network  technology  will  allow  very  localized  combined  arms  training  involving 
combat  platforms  and  aviation  assets  via  their  respective  virtual  reality  simulators  linked  into  a 
common  virtual  battle  space.  This  ultimate  system  is  illustrated  in  Figure  3. 
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Figure  2.  TTES  Equipment  Configuration  for  the  2002  Timeframe. 


Figure  3,  TTES  Equipment  Configuration  Planned  for  2006. 
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2.2  GOAL  OF  STUDY 

The  goal  of  the  present  study  is  to  determine  the  degree  to  which  TTES,  as  currently 
developed  and  envisioned,  will  be  of  value  to  those  numerous  military  and  civilian  agencies  that 
may  make  use  of  it  for  both  discretionary  decision  and  marksmanship  training.  A  detailed  survey 
of  a  limited  number  of  subject  matter  experts  was  carried  out  for  this  purpose  in  May  1995,  as  a 
follow-on  to  a  similar  survey  conducted  a  year  earlier  in  May  1994.4  A  comparison  of  the  results 
of  the  two  surveys  permits  a  rough  determination  of  the  rate  at  which  TTES  development  is 
progressing  towards  a  folly-satisfactory  system. 

2.3  STUDY  APPROACH 

Several  techniques  are  commonly  used  to  obtain  perceptions,  opinions,  and  judgments  from 
subject  matter  experts.  These  generally  fall  into  two  categories:  personal  interviews  and 
questionnaires. 

Personal  interviews  are  used  to  present  the  situation  verbally,  then  to  obtain  verbal  responses 
to  situations  and  questions.  Such  interviews  can  be  undirected  and  open-ended,  eliciting  those 
comments  that  happen  to  come  to  mind  and  that  the  expert  wishes  to  provide.  Interviews  also  can 
be  directed  and  structured,  with  the  interviewer  asking  a  prepared  set  of  questions  and  noting  the 
responses.  Interviews  are  useful  for  obtaining  opinions  when  the  possible  set  of  responses  is 
unknown  or  when  respondents  are  unlikely  to  take  the  time  to  complete  a  questionnaire.  They  are 
expensive  since  they  require  a  great  deal  of  time  on  the  part  of  the  interviewer,  and  results  may  be 
difficult  to  analyze. 

A  third  interview  technique  is  referred  to  as  the  verbal  protocol  method.  The  expert  actually 
uses  the  system  being  evaluated  (or  a  simulation  of  it),  and  verbally  provides  perceptions  and 
opinions  while  going  through  a  typical  scenario.  Like  other  interview  techniques,  the  verbal 
protocol  method  is  costly  to  use  and  results  are  hard  to  analyze.  Unlike  other  interview  techniques, 
this  methodology  requires  that  the  system  be  available  for  use  in  a  form  that  adequately  represents 
what  the  final  system  will  be  like.  The  TTES  prototype  system  is  still  in  its  development  stage; 
thus  the  use  of  this  technique  was  deemed  inappropriate  for  now. 

Questionnaires,  unlike  interviews,  require  that  the  surveyor  know  in  advance  what  information 
generally  is  sought.  Questionnaires  can  be  open-ended ;  these  are  used  to  obtain  opinions  when  the 
surveyor  does  not  wish  to  prejudice  the  expert  by  providing  a  prescribed  set  of  possible  responses. 
Open-ended  questionnaires  are  time-consuming  to  analyze,  but  sometimes  are  the  only  way  to 
obtain  needed  opinions. 

Structured  questionnaires  are  used  for  the  opposite  situation:  when  the  surveyor  has  prepared 
a  limited  set  of  responses  that  are  to  be  judged  in  some  maimer.  The  expert  then  may  be  asked  to 
select  one  or  more  responses,  to  rank  the  responses,  or  to  rate  them  on  a  scale  of  1  to  5  or  1  to  7. 
Such  questionnaires  must  be  thoughtfully  prepared  to  avoid  biasing  the  results.5  Once  the 


4  Naval  Air  Warfare  Center  Weapons  Division.  Team  Tactical  Engagement  Simulator  (TTES): 
Perceived  Training  Value,  by  J.H.  Lind  and  S.R.  Adams.  China  Lake,  Calif.,  NAWCWPNS,  December 
1994.  (NAWCWPNS  TM  7724) 

5  D.  Meister.  Behavioral  Analysis  and  Measurement  Techniques.  New  York,  John  Wiley  and  Sons, 
1985. 
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questionnaire  is  prepared,  questionnaire  administration  is  cost  effective  and  analysis  is 
straightforward. 

One  technique  that  uses  both  open-ended  and  structured  questionnaires  is  referred  to  as  the 
Delphi  method.6  Experts  first  respond  to  general  open-ended  questions,  within  provided  guidelines. 
Summary  statistics  are  generated  from  the  results.  These  are  used  to  prepare  a  second 
questionnaire  for  submission  to  the  experts,  for  further  refining  of  responses.  The  conventional 
Delphi  method  continues  iteration  of  questions  and  responses  until  the  response  distribution 
converges.  However,  Sackman  has  suggested  that  feeding  back  “correct”  responses  to  participants 
biases  the  results.7  Thus  two  iterations  usually  are  considered  adequate  for  most  studies. 

Cost  and  time  considerations  have  led  to  selection  of  the  modified  Delphi  method  described 
above  to  obtain  the  judgments  needed  for  this  study.  An  initial  open-ended  survey  in  1994 
provided  lists  of  potential  TTES  capabilities  that  might  be  considered  by  developers.  These  lists 
were  used  to  prepare  a  detailed  structured  questionnaire.  As  noted  above,  two  separate  surveys 
were  conducted  approximately  one  year  apart,  using  essentially  the  same  questionnaire  and 
participants  from  similar  backgrounds.  Details  of  the  application  of  the  method  are  provided  in 
Section  3.0.  The  questionnaire  used  for  data  collection  is  reproduced  in  Appendix  A. 

2.4  SCOPE  AND  VALIDITY  OF  STUDY 

2.4.1  Participant  Representativeness 

The  survey  discussed  in  this  report  makes  use  of  subject  matter  experts  to  obtain  judgments  of 
the  value  of  TTES  as  a  training  system.  Attempts  were  made  to  utilize  as  diverse  a  sample  of 
potential  TTES  users  as  possible.  However,  time  and  funding  constraints  have  bounded  the 
project,  and  the  use  of  a  limited  number  of  experts  from  several  activities  is  a  weakness  of  this 
study  (a  wider  phase  of  evaluations  will  be  conducted  between  July  1996  and  July  1997).  Twenty- 
eight  military  and  civilian  marksmanship  trainers  were  used  as  the  experts  in  1995;  21  others  had 
been  used  in  1994  (see  Sections  3.2  and  3.3).  These  individuals  primarily  were  drawn  each  year 
from  six  agencies  and  commands.  With  only  a  limited  number  of  experts  and  only  a  few  potential 
user  agencies  involved  in  the  survey,  the  sample  may  not  be  representative  of  the  total  potential 
TTES  user  population  and  may  be  biased  by  the  missions  of  those  surveyed. 

2.4.2  Combining  Military  and  Civilian  Responses 

1995  survey  respondents  have  been  divided  into  two  groups:  military  personnel  and  civilian 
law  enforcement  agents.  Questionnaire  responses  were  recorded  and  analyzed  separately  for  each 
group,  to  determine  whether  it  was  appropriate  to  combine  them.  As  reported  in  Section  5. 1 .2,  the 
general  order  of  option  selections  remained  relatively  constant  across  groups,  while  numerical 
rating  values  varied  somewhat.  Overall  average  ratings  from  the  civilian  and  the  military 
respondents  did  not  differ  in  any  significant  way.  Grouping  of  the  responses  is  not  expected  to  bias 
survey  results  substantially. 


6  N.  Dalkey  and  O.  Helmer.  “An  Experimental  Application  of  the  Delphi  Method  to  the  Use  of 
Experts.”  Management  Science,  Vol.  9  (1963). 

7  Rand  Corporation.  Delphi-Assessment:  Expert  Opinion,  Forecasting,  and  Group  Processes,  by  H. 
Sackman.  Santa  Monica,  Calif.,  Rand,  1974.  (Report  No.  R-1283-PR) 
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2.4.3  Validity  of  Civilian  Responses 

With  only  five  participants  in  the  civilian  group,  little  validity  can  be  implied  for  this  group’s 
responses,  when  considered  separately.  Potential  TTES  usefulness  and  current  capability  rating 
values  are  provided  in  this  report  for  the  five  civilians,  but  it  must  be  recognized  that  they  do  not 
represent  an  adequate  sample.  Civilian  results  are  reported  separately  in  Section  5  for 
completeness  and  so  differences  and  similarities  can  be  observed,  for  those  interested.  Although 
the  resulting  trends  may  be  useful,  great  caution  should  be  taken  in  making  any  hard  inferences 
from  this  individual  group’s  results. 

2.4.4  Participant  Responsiveness 

Questionnaires  used  for  data  collection  were  carefully  designed  to  be  as  comprehensive  as 
possible.  As  a  result,  they  were  quite  long  and  required  approximately  30  minutes  to  complete. 
Most  of  the  respondents  were  observed  while  completing  the  forms,  both  in  1994  and  1995,  and 
appeared  to  take  the  process  seriously  and  to  approach  the  task  thoughtfully.  Based  on  these 
observations,  we  feel  that  the  responses  are  valid  for  the  sample  that  was  surveyed. 

2.4.5  Comparisons  Between  1994  and  1995  Results 

Surveying  the  same  individuals  both  in  1994  and  1995  was  not  possible.  Only  those  invited  to 
attend  the  TTES  demonstrations  were  available  to  participate  in  the  surveys,  and  different  military 
personnel  and  civilians  were  selected  for  attendance  each  year.  Nonetheless,  survey  results 
obtained  in  1995  have  been  compared  with  those  from  the  1994  survey  to  obtain  a  rough  indication 
of  the  level  of  improvement  in  TTES  over  the  past  year. 

The  comparisons  have  some  credibility  since,  although  different  participants  were  used,  the  28 
1995  respondents  generally  came  from  the  same  kinds  of  military  and  civilian  communities  as  the 
21  1994  respondents.  In  1994,  62%  were  experienced  small  arms  trainers;  in  1995,  all 
respondents  were  involved  in  training  to  some  extent.  Civilians  made  up  14%  of  the  sample  in 

1994  and  18%  1995,  with  the  rest  drawn  from  various  military  facilities.  Military  ranks  (which 
generally  equate  to  experience  levels)  were  similarly  distributed  for  the  two  years. 

Substantially  higher  TTES  usefulness  and  capability  values  generally  were  assigned  by  the 

1995  participants.  Although  this  may  indicate  that  major  improvements  have  been  made  in  TTES 
in  the  past  year,  differences  may  be  the  result  (at  least  in  part)  of  using  different  samples  from  the 
target  populations,  and  relatively  small  samples  at  that.  Comparisons  of  the  results  from  the  two 
surveys  must  be  considered  approximate  and  very  general  in  nature. 

2.4.6  Capability  Ratings  for  Unimplemented  Factors 

The  questionnaire  asked  for  both  Potential  Usefulness  and  Current  Capability  ratings  for  the 
same  list  of  simulation  factors.  Some  of  the  listed  situations,  events,  and  objects  are  not  yet 
implemented  in  TTES,  even  in  rudimentary  form  (Trench  warfare.  Fighting  hole  tactics,  Arctic 
environment.  Bridges,  etc  ).  Thus  actual  observation  of  these  items  could  not  always  be  used  for 
judgments  of  Current  Capability. 

Participating  experts  were  instructed  to  rate  the  things  they  had  observed  in  TTES,  but  also 
were  invited  to  provide  judgments  on  things  they  had  not  actually  seen  but  could  imagine,  based  on 
the  quality  of  what  presently  is  available  in  TTES  and  on  descriptions  of  the  envisioned  system. 
Resulting  ratings  thus  represent  hypothesized  capability  levels,  indicating  how  good  the 
respondents  feel  these  unimplemented  TTES  capabilities  would  be,  if  they  were  present. 
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3.0  METHODOLOGY 


3.1  PRELIMINARY  1994  SURVEY 

As  prescribed  by  the  Delphi  method,  a  preliminary  open-ended  survey  was  conducted  in  early 
1994  to  elicit  general  information  on  the  kinds  of  capabilities  that  should  be  included  in  the  TTES 
system.  A  questionnaire  was  completed  by  a  group  of  3 1  subject  matter  experts  who  provided 
unconstrained  inputs  on  the  firing  positions,  individual  movements,  and  coordinated  tactical 
movements  they  would  like  to  see  as  part  of  TTES.  Information  on  head,  eye,  arm,  and  hand 
movements  also  was  elicited,  along  with  the  objects  that  should  be  present  in  the  simulations. 
Training  situations,  shooting  events,  etc.,  also  were  included  by  this  initial  group  of  respondents. 

3.2  COMPREHENSIVE  1994  SURVEY 

Continuing  with  the  Delphi  method,  results  of  the  preliminary  survey,  along  with  verbal 
inputs  from  experts,  were  used  to  develop  a  comprehensive  12-page  structured  questionnaire.  The 
questionnaire  was  prepared  according  to  the  guidelines  of  the  US.  Army  Questionnaire 
Construction  Manual 8  and  administered  to  2 1  individuals  participating  in  a  demonstration  of  the 
TTES  system  in  May  1994.  Of  those  subject  matter  experts  surveyed  in  1994,  13  were 
experienced  weapons-use  trainers  (including  three  civilian  law  enforcement  officers),  while  eight 
were  U  S.  Marine  Corps  Military  Police  who  had  received  extensive  small  arms  training.  The 
study  participants  used  the  then-current  version  of  TTES  and  received  a  briefing  on  the  capabilities 
the  ultimate  system  will  have,  prior  to  completing  the  detailed  survey  form. 

The  1994  questionnaire  asked  respondents  to  rate  the  Importance  of  various  factors  making 
up  15  categories  of  simulation  capabilities  that  might  be  included  in  TTES  simulations.  Categories 
included  situations,  events,  environments,  objects,  humans,  weapons,  sounds,  and  tactical  and  body 
positions  and  movements.  The  importance  of  including  each  category,  per  se,  was  not  rated, 
because  inclusion  of  all  categories  will  be  critical  for  a  complete,  comprehensive  system. 

However,  information  on  the  relative  value  of  each  factor  within  a  category  was  needed  to  set 
TTES  development  priorities.  Respondents  also  gave  ratings  for  perceived  TI  ES  capability  to 
emulate  these  factors  and  categories,  both  in  the  system’s  1994  form  ( Current  Capability)  and  as 
TTES  is  planned  when  the  project  is  completed  ( Potential  Capability).  A  5-point  rating  scale  was 
used  to  indicate  levels  of  importance  and  capability. 

The  1994  respondents  generally  gave  very  high  ratings  for  the  importance  of  including  a  wide 
variety  of  factors  in  TTES  for  system  realism.  Based  on  ratings  assigned  by  the  total  group  of 
participants,  factors  were  classified  as  critical  to  include  in  TTES,  extremely  important  to  include, 
important  to  include,  and  consider  including  as  funds  permit.  From  among  the  15  categories 
included  on  the  survey,  respondents  indicated  that  Simulated  Adversaries  (92.7%)  and  Sounds 
(86.7%)  were  the  most  important  categories  of  factors  needed  in  TTES.  Distances  between 
adjacent  ratings  were  small;  even  the  bottom-rated  item.  Eye  Movements,  received  a  calculated 
rating  of  74.6%  in  importance  for  TTES. 

The  ratings  indicated  that  the  experts  considered  it  extremely  important  that  the  final  TTES 
system  not  constrain  the  activities  of  its  users  any  more  than  absolutely  necessary.  Equipment 
used  by  trainees  should  allow  them  to  carry  out,  in  a  normal  manner,  those  procedures  and 

8  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  Questionnaire  Construction 
Manual,  by  B.A.  Babbitt  and  C  O.  Nystrom.  Ft.  Hood,  Tex.,  ARI  Field  Unit,  June  1989.  (Research 
Product  89-20) 
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activities  that  are  common  during  combat.  On  the  other  hand,  although  the  Simulated  Adversaries 
category  was  the  highest  rated  in  importance,  high-fidelity  simulation  of  the  humans  who  serve  as 
adversaries  is  considered  less  important.  Various  factors  in  the  Arm  &  Hand,  Head,  and  Eye 
Movements  categories  were  rated  lower  than  factors  in  many  other  categories. 

Current  Capability  was  rated  relatively  low  for  TTES  in  1994  —  not  surprising  since 
development  had  not  progressed  far.  The  overall  calculated  total  group  rating  was  42%,  while  the 
subgroup  of  13  trainers  gave  a  calculated  rating  of  only  39%. 

The  most  significant  1994  finding  concerned  TTES’s  Potential  Capability,  ratings  indicated 
that  the  experts  felt  TTES  will  provide  very  high  quality  training  that  can  meet  between  80  and 
85%  of  their  perceived  needs  for  discretionary  decision  and  small  arms  training.  This  finding 
suggests  that  a  simulator  system  such  as  TTES  eventually  could  result  in  phenomenal  savings  in 
time,  ammunition,  and  other  costs  related  to  training,  while  providing  the  capability  to  train  to 
higher  standards  and  to  challenge  trainees  in  ways  not  currently  possible. 

3.3  1995  SURVEY 

The  survey  conducted  in  May  1995  included  all  15  categories  of  simulation  capabilities  from 
the  1994  questionnaire,  along  with  the  same  lists  of  simulation  factors.  Two  additional  survey 
categories  were  added  to  query  respondents  on  how  useful  and  capable  TTES  is  (or  will  be)  for 
Mission  Preview  and  Mission  Rehearsal  operations  (a  separate  survey  also  was  used  to  gather  data 
for  these  categories;  see  Appendix  B).  Again,  Current  Capability  judgments  were  solicited. 
However,  instead  of  obtaining  separate  judgments  on  Potential  Capability  and  Importance,  the 
term  Potential  Usefulness  was  used  to  combine  the  two  concepts.  This  was  done  to  shorten  the 
survey  form  slightly  while  obtaining  essentially  the  same  data. 

Each  of  the  17  questionnaire  categories  was  made  up  of  6  to  27  separate  factors  that  might  be 
modeled.  For  example,  the  TTES  Training  Situations  category  included  Anti-armor  missions, 
Immediate  action  drills,  MOUT patrolling  operations,  etc.  An  Other  option  was  included  for 
additional  factors  that  might  occur  to  respondents  (but  very  few  suggestions  were  made).  For  each 
individual  factor,  respondents  were  asked  to  indicate  a  Potential  Usefulness  rating  and  a  Current 
Capability  rating.  In  addition,  respondents  were  asked  to  provide  their  perceptions  of  TTES 
Current  Capability  and  Potential  Usefulness  for  each  category  as  a  whole. 

Responses  were  provided  on  a  linear  5 -point  scale.  Potential  Usefulness  was  rated  from  1 
(very  useful)  to  5  (very  useless).  These  judgments  can  assist  in  prioritizing  the  order  in  which 
simulation  categories  and  factors  are  addressed  during  TTES  development.  Current  Capability 
was  rated  on  a  scale  from  1  (very  good)  to  5  (very  poor).  Results  can  be  used  to  evaluate  how  far 
along  trainers  perceive  TTES  to  be,  as  development  of  system  usefulness  and  realism  continues. 
Factors  perceived  as  close  to  adequate  can  receive  less  attention  during  continued  development  than 
those  judged  to  be  lacking  in  realism  and  requiring  substantial  further  work. 

Also  added  to  the  1995  survey  form  were  questions  on  the  Realism  Level  needed  for 
satisfactory  training  (while  keeping  costs  down).  How  good  is  “good  enough”?  Required  realism 
ratings  were  asked  only  for  the  section  of  the  survey  that  concerned  TTES  objects,  situations,  and 
events  (Section  B).  Again,  respondents  used  a  5-point  linear  scale.  The  categories  were  as  follows 
(general  terms  are  used  here;  see  Appendix  A  for  the  exact  wording  used  for  each  question). 

1.  Exact  replication  of  real-world  objects,  situations,  and  events. 
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2.  Good  replication,  hard  to  tell  it  is  not  real. 

3.  Reasonable  replication,  appearance  about  as  expected  for  this  kind  of  object,  situation,  or 
event. 

4.  So-so  replication,  appearance  somewhat  realistic. 

5.  Crude  approximation,  just  enough  to  recognize  the  object,  situation,  or  event. 

3.4  STUDY  PARTICIPANTS 

Twenty-eight  male  military  and  civilian  personnel  participated  in  the  1995  survey.  The 
following  agencies  were  represented  (see  Figure  4): 

•  U  S.  Marine  Corps  Quantico  Weapon  Training  Battalion  (WT  Bn),  Quantico,  Virginia:  10 

•  Marine  Corps  Security  Forces  (MCSF)  School,  Chesapeake,  Virginia:  8 

•  MCSF  Battalion  Fleet  Anti-Terrorist  Security  Team  (FAST),  Norfolk,  Virginia:  5 

•  Federal  Bureau  of  Investigation  (FBI)  Academy,  Quantico,  Virginia:  2 

•  San  Antonio  (Texas)  Police  Department:  2 

•  Los  Angeles  (California)  SherifFs  Department:  1 


Agency  Affiliations 


0%  5%  10%  15%  20%  25%  30%  15%  40% 


Figure  4.  Agency  Affiliations  of  the  28  Survey  Respondents. 

Of  the  participants,  23  (  82%)  were  military  and  five  (18%)  were  civilians.  Figure  5  shows 
the  years  of  training  experience  reported  by  the  experts.  All  military  participants  have  infantry  or 
security  backgrounds  and  are  responsible  for  weapons  or  tactics  training.  The  Weapons  Training 
Battalion  teaches  marksmanship.  MCSF  personnel  train  Marines  who  guard  state  embassies,  etc. 
FAST  personnel  train  those  responsible  for  security  and  anti-terrorist  missions  at  various  vital 
installations. 


As  discussed  in  Section  2.4.5,  the  participants  in  the  1995  survey  were  not  the  same 
individuals  as  in  1994,  but  the  28  1995  respondents  generally  came  from  the  same  military  and 
civilian  communities  as  the  21  1994  respondents.  In  1994,  62%  were  experienced  small  arms 
trainers;  in  1995,  all  respondents  were  involved  in  training  to  some  extent.  In  1994,  14%  of  the 
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participants  were  civilian  law  enforcement  officers  while  in  1 995  civilians  made  up  1 8%  of  the 
sample;  the  rest  were  drawn  from  various  Marine  Corps,  Army,  and  Air  Force  communities. 
Military  ranks  (which  generally  equate  to  experience  levels)  were  similarly  distributed  for  the  two 
years. 


Trainer  Experience  Levels 


2  -  5  Yrs 
25% 


Figure  5.  Levels  of  Experience  as  Trainers  Reported  by  Respondents. 

3.5  SURVEY  ADMINISTRATION 

The  1994  and  1995  TTES  evaluations  were  both  carried  out  at  the  Marine  Corps  MOUT II 
Quantico  Combat  Training  Village,  near  Quantico,  Virginia,  in  conjunction  with  a  demonstration 
of  the  TTES  system.  TTES  engineers  had  generated  a  virtual  representation  of  the  MOUT  II 
facility  on  the  training  devices.  Thus  individuals  participating  in  the  assessment  were  able  to  make 
tactical  observations  concerning  the  TTES  MOUT  representation  and  the  actual  site,  along  with 
comparisons  between  the  virtual  and  the  real  areas. 


In  1994,  only  one  TTES  system  was  available  for  evaluation,  limiting  scenarios  significantly. 
In  1995,  two  one-man  TTES  systems  were  set  up  in  a  warehouse  on  the  site.  Survey  participants 
gathered  there  for  a  briefing  on  the  current  TTES  system  and  on  system  capabilities  that  are 
anticipated  when  the  simulator  project  is  completed.  The  weapon  used  for  the  TTES 
demonstrations  and  evaluations  was  a  demilitarized  but  fully  functional  M16-A2  rifle.  Participants 
controlled  the  area  that  was  viewed  on  the  display  screen  via  a  foot  pedal  that,  when  depressed, 
resulted  in  apparent  movement  through  the  simulated  scene.  The  side-to-side  view  was  controlled 
by  direction  of  gaze,  monitored  via  a  head-position  tracker  contained  in  a  helmet. 


Simulated  two-man  patrol  team  scenarios  were  used  in  1995,  taking  advantage  of  the  two 
TTES  systems.  The  systems  were  situated  so  that  the  two  participants  could  not  see  each  other  in 
the  room.  Instead,  each  observed  a  simulated  on-screen  trooper  representing  the  other  team 
member  as  he  would  appear  when  located  in  a  position  that  would  be  visible  in  the  actual  MOUT 
training  village.  Communications  were  via  microphones  and  headphones  in  the  head-position 
tracker  helmets.  Thus  coordinated  actions  could  be  carried  out  as  in  the  real  world. 
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Exercises  lasted  about  20  minutes  and  consisted  of  moving  through  the  terrain  and  buildings 
as  desired.  TTES  users  were  given  the  opportunity  to  traverse  the  virtual  village’s  open  areas  and 
roads  and  to  explore  the  buildings,  including  a  representation  of  the  actual  MOUT II  hotel 
building.  This  three-story  building  was  simulated  in  detail,  including  rooms,  corridors,  stairs, 
doorways,  and  windows.  Participants  were  encouraged  to  move  freely  throughout  the  building  and 
to  become  familiar  with  its  layout. 

During  the  exercises  participants  encountered  computer-controlled  hostiles:  semi-realistic 
emulations  of  uniformed  nfle-bearing  enemy  soldiers  who  recognized  trainee  presence  and  fired 
weapons  as  appropriate.  Hostiles  were  engaged  as  encountered  and  the  participants  had  the 
opportunity  to  try  their  decision-making  and  marksmanship  skills.  Simplistic  representations  of 
neutrals  also  were  included  in  1995  (but  not  in  1994):  smaller  men  wearing  red  shirts  and  blue 
trousers,  who  appeared  in  the  distance,  hurried  across  the  scene,  and  presented  no  obvious  threat. 

Not  all  survey  respondents  actually  used  the  TTES  system,  but  all  observed  other  participants 
going  through  the  scenarios.  After  the  exercises,  each  respondent  was  given  a  copy  of  the  survey 
form,  was  verbally  instructed  about  what  was  requested,  and  was  told  that  his  inputs  would  be 
used  to  guide  the  continued  development  of  TTES.  Respondents  were  allowed  as  much  time  as 
desired  to  complete  the  form,  but  most  completed  it  in  about  30  minutes. 

3.6  DATA  ANALYSIS  AND  INTERPRETATION 

The  1995  survey  results  (which  are  the  focus  of  this  report)  consisted  of  Potential  Usefulness, 
Current  Capability,  and  required  Realism  Level  ratings  that  ranged  from  1  to  5,  assigned  to  the 
individual  simulation  factors  by  survey  participants.  Data  were  compiled  from  the  28  survey 
forms  and  entered  into  the  Microsoft  Excel  spreadsheet  program.9  Results  were  calculated  for  the 
group  of  experts  as  a  whole  and  also  separately  for  the  subgroups  of  military  and  civilian 
participants. 

The  5 -point  numerical  ratings  assigned  by  the  subject  matter  experts  were  considered  to  be 
interval-level  data  due  to  the  manner  of  questionnaire  construction.  As  a  result,  the  individual 
respondents’  values  could  be  combined  and  normalized  onto  a  100-point  scale.  The  combined 
group  ratings  were  calculated  for  Potential  Usefulness,  for  Current  Capability,  and  for  Realism 
Level.  This  was  done  for  the  total  group,  the  group  of  military  respondents,  and  the  group  of 
civilian  respondents. 

Calculated  ratings  were  determined  for  each  of  the  individual  factors  that  made  up  each  of  the 
17  questionnaire  categories.  For  each  factor,  the  ratings  were  converted  to  a  100-point  scale 
(higher  is  better)  by  (1)  multiplying  each  individual’s  rating  of  1  by  5,  each  2  by  4,  etc.,  (2) 
summing  the  resulting  values  for  all  responses  to  that  factor  question,  (3)  dividing  the  sum  by  the 
number  of  listed  responses  to  that  question  multiplied  by  5  (the  maximum  value  possible  for  that 
factor),  and  (4)  multiplying  the  result  by  100.  Thus  the  ratings  represent  a  percent,  based  on  the 
ratio  of  the  value  assigned  to  the  maximum  possible  value  that  could  have  been  assigned  by  the 
group  for  an  individual  factor.  These  results  are  included  in  Section  5.0. 


9  Sincere  appreciation  is  expressed  to  Therese  Bilodeau,  NPS  Operations  Research  Department 
Editorial  Assistant,  who  compiled  the  survey  results  and  entered  them  into  the  Excel  spreadsheets. 
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Trends  in  the  data  (that  is,  rank  orderings)  were  determined  instead  of  making  direct 
comparisons  of  adjacent  ratings,  due  to  the  small  sample  available  for  the  survey:  there  is  no 
statistically-significant  difference  between  a  calculated  Potential  Usefulness  rating  of  88. 1%  for 
Anti-armor/artillery  missions  and  one  of  85.7%  for  Immediate  action  drills.  However,  the 
observed  trends  are  important.  When  Care  for  wounded  receives  a  calculated  overall  Potential 
Usefulness  rating  of  66.4%,  it  clearly  is  considered  less  important  than  Building  clearing  with  a 
rating  of  96.4%. 

How  should  the  various  Potential  Usefulness  responses  be  interpreted?  The  subject  matter 
experts  who  participated  in  this  study  were  highly  experienced,  and  their  opinions  should  be  given 
considerable  weight.  Thus  we  recommend  the  following  Potential  Usefulness  rating 
classifications. 


Potential  Usefulness  for  TTES  Training 

Calculated  Group  Ratings 

Critical  to  include  in  TTES 

>  89.9% 

Extremely  important  to  include 

80.0  -  89.9% 

Important  to  include 

70.0  -  79.9% 

Consider  including,  as  funds  permit 

60.0  -  69.9% 

Probably  not  important  to  include 

<  60.0% 

Current  Capability  ratings  also  require  some  interpretation.  For  purposes  of  this  study,  items 
rated  at  less  than  50%  were  considered  generally  poorly  modeled  (and  sometimes  have  not  been 
modeled  at  all).  Higher  ratings  obviously  indicated  greater  satisfaction  with  the  present  TTES 
modeling  of  a  simulation  factor  —  but  the  calculated  numerical  capability  ratings  are  of  less 
usefulness  than  the  ranking  of  items,  due  to  the  small  sample  size. 

1995  Total  Group  Current  Capability  ratings  were  directly  compared  with  1994  ratings  for 
the  same  items  (recognizing  the  validity  problems  noted  in  Section  2.4.5)  to  obtain  point  and 
percentage  differences  between  the  two  years.  The  results  are  used  here  as  an  indication  of 
modeling  progress  over  the  past  year,  as  perceived  by  the  groups  of  survey  experts. 

Required  Realism  Level  ratings  were  interpreted  much  like  the  Potential  Usefulness  and 
Current  Capability  ratings.  Items  with  calculated  percentages  of  80%  or  better  are  considered  to 
require  Exact  replication;  between  60  and  80%,  Good  replication;  between  40  and  60% 

Reasonable  replication,  etc.  (see  Section  3.3). 


16 


4.0  RESPONSES  TO  SURVEY  QUESTIONS 


Results  are  reported  here  in  the  same  order  and  with  the  same  section  headings  as  on  the 
survey  form  (Appendix  A).  Factors  that  are  or  possibly  could  be  simulated  in  TTES  have  been 
grouped  into  17  general  categories.  These  categories  then  are  arranged  under  three  major 
headings:  (1)  overall  TTES  usefulness;  (2)  TTES  objects,  situations,  and  events;  and  (3)  trainee 
and  adversary  movements  and  positions. 

Respondents  rated  the  Potential  Usefulness  of  each  factor  and  also  provided  an  overall 
judgment  on  the  usefulness  of  the  entire  category  of  simulation  factors.  They  also  rated  the 
Current  Capability  of  TTES  to  emulate  each  event,  situation,  or  object  and  provided  judgments  on 
overall  TTES  Current  Capability  to  emulate  each  category  of  factors  as  a  whole  (see  Section 
2.4.6).  In  addition,  for  Section  B  only,  ratings  for  the  required  Realism  Level  for  each  category 
also  were  requested. 

The  following  sections  provide  tables  that  list  group  ratings  for  the  Potential  Usefulness  of 
each  factor  within  each  of  the  survey  categories,  for  TTES  Current  Capability  to  emulate  that 
factor,  and  for  the  required  Realism  Level  (when  applicable).  Each  of  the  questions  is  stated 
exactly  as  posed  to  the  survey  participants  (shown  in  boxes).  Ratings  are  provided  for  the  Total 
Group  (the  28  participants  as  a  whole),  and  also  individually  for  the  group  of  23  Military 
participants  and  the  five  Civilian  law  enforcement  officers. 

Separate  tables  are  provided  for  Potential  Usefulness  and  Current  Capability  for  each 
category  of  simulation  factors.  Within  each  table,  the  factors  have  been  sorted  according  to  the 
Total  Group’s  ratings,  with  the  factor  considered  most  useful  or  most  capably  modeled  at  the  top. 
Total  Group  Current  Capability  responses  from  the  1994  survey  also  are  included  in  the  tables  for 
comparison  with  the  1995  results.  A  rough  determination  is  made  of  the  level  of  improvement  over 
the  past  year,  based  on  respondents’  judgments. 

Some  discussion  of  the  results  is  included  in  the  following  sections  (primarily  for  Total  Group 
responses);  most  results  are  unsurprising  and  easy  to  interpret.  Overall  judgments  of  TTES 
Current  Capability,  Potential  Usefulness,  and  Realism  Level  are  provided  at  the  bottom  of  the 
tables  but  generally  are  discussed  only  minimally  in  this  section;  these  are  covered  in  more  detail 
in  Section  5.  However,  it  should  be  noted  that  these  Overall  ratings  are  not  simply  averages  of  the 
above-listed  individual  factor  ratings.  Instead,  these  are  separate  ratings  provided  by  the 
respondents  as  judgments  of  the  usefulness  and  capability  of  the  category  itself. 

4.1  OVERALL  TTES  USEFULNESS 

Section  A  of  the  1995  survey  concerned  the  overall  structure  and  usefulness  of  the  system. 
Questions  were  asked  concerning  (1)  the  training  situations  that  presently  are  included  and  those 
that  eventually  will  be,  (2)  training  events  related  to  marksmanship,  (3)  TTES  events  related  to 
making  decisions  about  how  to  respond  to  possible  adversaries,  (4)  capabilities  TTES  should 
provide  for  previewing  missions,  and  (5)  capabilities  needed  to  use  TTES  for  mission  rehearsals. 


10  Results  of  a  separate  survey  related  to  using  TTES  for  mission  preview  and  rehearsal,  also 
conducted  this  year,  are  included  in  Appendix  B. 
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4.1.1  Training  Events 

1 .  TTES  training  situations.  What  training  situations  should  be  included  in  the  TTES 

scenarios  and  synthetic  environment,  for  maximum  TTES  usefulness  as  a  training  technology? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  Nine  of  the  listed  Training  situations  are  considered  critical  to  include  in  TTES,  based  on 
calculated  Potential  Usefulness  ratings  of  90%  or  better  for  the  Total  Group.  Top-rated  were 
Building  and  Room  clearing,  along  with  Close-quarter  battle,  though  differences  among  the  top 
factors  should  not  be  considered  statistically  significant. 


Potential  Usefulness  Rating 


TTES  Training  Situation 


Building  clearing 


Room  clearing 


Close-quarter  battle 


Advance  through  urban  areas 


Combat  rehearsals 


MOUT  patrolling  operations 


Squad  &  fire  team  tactics 


Sniping  activities  _ 


Small  unit  leadership  practice 


Hostage  rescue  team  activities 


Anti-armor  artillery  missions 


Raid  operations 


Advance-thru-terrain  activities 


Perimeter  defense 


Combined  arms  operations 


Immediate  action  drills 


Recon  operations 


Ambush  operations 


Unit-to-unit  coordination 


Obstacle  breaching  tactics 


Trench  warfare 


Demolition/EOD  operations 


Fighting  hole  tactics 


Land  navigation  practice 


Guard  duty 


Neutral  evacuation  operations 


Care  for  wounded 


Total  Group 


Military 


Civilians 


95.7 

100.0 

94.8 

100.0 

94.5 

96.0 

92.0 

92.0 

89.5 

100.0 

90.5 

95.0 

89.6 

100.0 

89.6 

96.0 

89.1 

96.0 

87.0 

96.0 

86.7 

100.0 

86.1 

92.0 

86.7 

88.0 

83.8 

100.0 

83.6 

96.0 

84.3 

92.0 

Potential  Overall  TTES 
usefulness:  Training  Situations 


Thirteen  factors  were  rated  between  80  and  90%,  classified  by  the  total  group  as  extremely 
important  factors  to  include. 


Four  of  the  listed  factors  were  in  the  70  to  80%  range,  and  fell  in  the  important  to  include 
classification. 
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Only  Care  for  wounded  rated  below  70%;  this  factor  fell  in  the  consider  including  category. 


•  The  five  Civilians  gave  ratings  to  the  individual  factors  that  were  equal  to  or  higher  than  those 
of  the  23  military  personnel  —  with  the  exception  of  Fighting  hole  tactics. 

•  The  total  group’s  1995  rating  for  Potential  Usefulness  of  these  factors  is  96.4%;  in  1994,  the 
same  factors  received  an  average  Importance  rating  of  only  76.4%  (see  section  5.1).  This 
represents  significant  increase  during  the  past  year  in  the  perceived  usefulness  or  importance  of 
representing  a  variety  of  Training  Situations  in  TTES. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  All  but  one  of  the  listed  Training  Situations  (Demolition/EOD  operations)  received  higher 
Total  Group  Current  Capability  ratings  in  1995  than  in  1994. 


Current  Capability  Rating 

TTES  Training  Situation 

Total  Group 

Military 

Civilians 

1994  Total 
Group 

Building  clearing 

75.7 

73.7 

85.0 

52.0 

Advance  through  urban  areas 

75.0 

76.3 

70.0 

54.4 

Room  clearing 

75.0 

74.0 

80.0 

54.0 

MOUT  patrolling  operations 

74.8 

76.0 

66.7 

60.0 

Combat  rehearsals 

70.0 

68.3 

75.0 

48.4 

Squad  &  fire  team  tactics 

69.4 

70.0 

66.7 

40.0 

Close-quarter  battle 

69.1 

68.9 

70.0 

51.0 

Small  unit  leadership  practice 

68.0 

67.3 

70.0 

41.1 

Recon  operations 

64.6 

60.0 

75.0 

42.2 

Guard  duty 

61.8 

65.0 

53.3 

43.3 

Advance-thru-terrain  activities 

61.5 

57.8 

70.0 

50.0 

Sniping  activities 

61.3 

60.0 

65.0 

43.3 

Raid  operations 

58.7 

60.0 

55.0 

48.0 

Perimeter  defense 

58.2 

52.5 

73.3 

43.3 

Ambush  operations 

55.7 

50.0 

70.0 

49.5 

Unit-to-unit  coordination 

55.0 

53.3 

60.0 

37.8 

Combined  arms  operations 

54.7 

49.1 

70.0 

46.7 

Hostage  rescue  team  activities 

54.3 

54.0 

55.0 

51.0 

Immediate  action  drills 

53.7 

49.3 

70.0 

35.6 

Land  navigation  practice 

53.3 

53.3 

53.3 

38.9 

Neutral  evacuation  operations 

49.1 

46.7 

60.0 

42.2 

Trench  warfare 

46.7 

42.9 

60.0 

36.7 

Fighting  hole  tactics 

42.2 

42.9 

40.0 

36.5 

Care  for  wounded 

42.0 

40.0 

46.7 

34.4 

Obstacle  breaching  tactics 

41.8 

40.0 

46.7 

35.6 

Anti-armor  artillery  missions 

38.0 

35.0 

50.0 

35.6 

Demolition/EOD  operations 

35.6 

34.3 

40.0 

36.5 

Current  Overall  TTES  capability: 
Training  Situations 

69.5 

71.8 

60.0 

45.0 

•  The  Total  Group’s  Overall  Training  Situations  category  rating  in  1995  was  almost  25  points 
higher  than  in  1994,  indicating  general  improvements  of  more  than  50%. 


19 


•  Greatest  improvement  appears  to  be  in  Squad  and  fire  team  tactics  (29%)  and  in  Small  unit 
leadership  practice  (27%).  The  ability  to  train  using  a  two-man  scenario  is  seen  as  a  major 
enhancement  to  TTES. 

•  In  spite  of  improvements,  seven  factors  still  received  Current  Capability  ratings  of  less  than 
50%:  Neutral  evacuation  operations.  Trench  warfare,  Fighting  hole  tactics,  Care  for 
wounded,  Obstacle  breaching  tactics.  Anti-armor  artillery'  missions,  and  Demolition/EOD 
operations.  Of  these,  all  but  Anti-armor  artillery  missions  also  were  ranked  among  the  lowest 
in  Potential  Usefulness.  Modeling  of  these  factors  for  TTES  has  received  little  or  no  attention 
so  far,  since  other  situations  are  more  important  for  general  training. 

4.1.2  Marksmanship 

2.  TTES  events:  marksmanship.  What  events  should  be  included  in  the  TTES  scenarios  and 
synthetic  environment  to  aid  in  marksmanship  training? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  All  but  two  of  the  17  Marksmanship  Events  were  rated  at  90%  or  above  in  Potential 
Usefulness  by  the  Total  Group,  placing  them  in  the  critical  to  include  category. 


Potential  Usefulness  Rating 

Marksmanship  Events 

Total  Group 

Military 

Civilians 

Quick  aim/fire 

97.1 

96.5 

100.0 

Indoor  shots 

96.4 

95.7 

100.0 

Fire  at  moving  targets 

95.0 

93.9 

100.0 

Close-range  shots 

94.8 

94.8 

95.0 

Track  moving  targets 

94.3 

93.0 

100.0 

Reload  weapon 

93.6 

93.0 

96.0 

Shoot  while  moving 

93.6 

92.2 

100.0 

Change  weapons  (rifle/pistol) 

92.8 

91.8 

100.0 

Targets  at  unknown  ranges 

92.6 

92.2 

95.0 

Multiple  targets,  multiple  shots 

92.3 

91.8 

95.0 

Awkward  shooting  positions 

92.1 

91.3 

96.0 

Precision  shots  at  small  targets 

91.9 

91.8 

92.0 

Clear  weapon  malfunctions 

91.1 

90.0 

96.0 

Sustained  rate  of  fire 

91.1 

89.6 

100.0 

Long-range  shots 

90.4 

89.6 

95.0 

Sniping 

87.7 

84.8 

100.0 

Throw  grenades 

87.2 

85.7 

95.0 

Potential  Overall  TTES  usefulness: 
Marksmanship  Events 

95.0 

95.0 

95.0 

•  The  remaining  two  simulation  factors.  Sniping  and  Throw  grenades,  each  rated  at  more  than 
87%  by  the  total  group,  should  be  considered  extremely  important  to  include  in  TTES. 

•  The  five  Civilians  rated  the  Potential  Usefulness  of  all  of  the  individual  Marksmanship  Events 
higher  than  the  Military  respondents,  but  their  Overall  rating  for  the  category  as  a  whole  was 
the  same. 
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Potential  Usefulness  of  the  Marksmanship  Events  category  was  rated  10  points  higher  in  1995 
(95.0%)  than  was  their  Importance  in  1994  (85.0%)  by  the  Total  Group  of  respondents. 


Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  All  of  the  individual  listed  Marksmanship  Events  received  higher  Total  Group  Current 
Capability  ratings  in  1995  than  in  1994. 


Current  Capability  Rating 

Marksmanship  Events 

Total  Group 

Military 

Civilians 

1994  Total 
Group 

Reload  weapon 

77.7 

75.2 

88.0 

36.8 

Indoor  shots 

76.7 

73.7 

88.0 

56.8 

Close-range  shots 

69.6 

70.4 

65.0 

56.0 

Quick  aim/fire 

68.6 

69.6 

64.0 

55.0 

Sustained  rate  of  fire 

67.3 

66.7 

70.0 

45.3  1 

Fire  at  moving  targets 

65.6 

68.0 

56.0 

43.0 

Shoot  while  moving 

65.2 

68.9 

52.0 

42.1 

Track  moving  targets 

63.3 

63.2 

64.0 

40.0 

Multiple  targets,  multiple  shots 

63.2 

66.7 

50.0 

45.3 

Targets  at  unknown  ranges 

63.1 

65.5 

50.0 

50.0 

Awkward  shooting  positions 

61.9 

61.3 

64.0 

40.0 

Sniping 

60.0 

62.7 

52.0 

39.0 

Long-range  shots 

60.0 

62.6 

45.0 

49.0 

!  Clear  weapon  malfunctions 

57.6 

55.0 

64.0 

34.7 

Precision  shots  at  small  targets 

56.8 

59.0 

48.0 

51.0 

Change  weapons  (rifle/pistol) 

44.6 

48.0 

33.3 

31.6 

Throw  grenades 

34.5 

37.5 

26.7 

33.3 

Current  Overall  TTES  capability: 
Marksmanship  Events 

71.1 

71.4 

70.0 

46.3 

•  The  Overall  Marksmanship  Events  category  was  rated  almost  25  points  higher  in  Current 
Capability  in  1995,  indicating  a  perceived  improvement  of  more  than  50%. 

•  Reload  weapon  received  the  greatest  jump  in  rating,  over  40  points,  representing  an 
improvement  of  over  100%. 

•  Ratings  for  seven  other  Marksmanship  Events  were  up  20  points  or  more:  Sustained  rate  of 
fire.  Fire  at  moving  targets,  Shoot  while  moving.  Track  moving  targets.  Awkward  shooting 
positions.  Sniping,  and  Clear  weapon  malfunction. 


Only  two  factors  still  were  rated  below  50%  in  Current  Capability.  Change  weapons 
(rifle/pistol)  and  Throw  grenades.  Neither  capability  has  been  modeled  so  far  (see  Section 
2.4.6),  though  survey  participants  consider  that  both  are  important  to  implement  in  TTES. 


4.1.3  Discretionary  Decisions 


3.  TTES  events:  discretionary  decisions.  What  events  should  be  included  in  the  TTES 
scenarios  and  synthetic  environment  to  aid  in  making  decisions  about  responses  to  possible 
adversaries? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  Nine  of  the  listed  factors  received  calculated  Total  Group  Potential  Usefulness  ratings  of  90% 
or  higher,  placing  them  in  the  cnrtca/-to-include  category. 


Potential  Usefulness  Rating 

Discretionary  Decisions 

Total  Group 

Military 

Civilians 

Snipers  in  area 

96.4 

95.7 

100.0 

Individual  hostiles  in  area 

95.0 

94.8 

96.0 

Friendlies/Allied  troops  in  area 

95.0 

94.8 

96.0 

Threatening  weapon  presentation 

94.3 

95.7 

88.0 

Hostaqes  in  area 

93.6 

93.9 

92.0 

Unknowns  in  area 

92.9 

93.0 

92.0 

Villagers/bystanders  in  area 

92.9 

92.2 

96.0 

Fleeing  enemy 

92.1 

93.0 

88.0 

Enemy  prisoners  of  war 

90.0 

90.4 

88.0 

Hostile  crowds 

87.9 

88.7 

84.0 

Non-threatening  wpn  presentation 

87.9 

88.7 

84.0 

Wounded  enemy 

87.9 

87.0 

92.0 

Enemies  in  Allied  uniforms 

86.4 

87.0 

84.0 

Potential  Overall  TTES  usefulness: 
Discretionary  Decisions 

95.2 

95.3 

95.0 

•  The  remaining  four  factors  were  rated  better  than  80%  by  the  Total  Group  and  also  by  both 
Military  and  Civilian  groups  separately,  and  should  be  considered  extremely  important  to 
include  in  TTES. 

•  In  1994,  this  group  of  factors  was  rated  83.5%  in  average  Importance  for  TTES.  This  year’s 
still  higher  Potential  Usefulness  rating  of  95.2%  may  indicate  that,  as  modeling  has  improved, 
TTES  users  feel  that  it  is  even  more  important  to  include  a  reasonable  range  of  situations  and 
computer-controlled  battlefield  entities  for  decision-making  training. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Every  listed  Discretionary  Decision  factor  received  a  higher  Total  Group  Current  Capability 
rating  in  1995  than  in  1994. 

•  Greatest  improvement  (20  points  or  more  difference)  was  noted  for  Individual  hostiles, 
Friendlies/ Allied  troops.  Unknowns,  Villagers,  and  Fleeing  enemies.  Friendlies,  Unknowns, 
and  Villagers  had  not  been  modeled  pnor  to  the  1994  survey,  so  it  is  not  surprising  that 
between  68%  and  75%  improved  capability  was  perceived  in  1995.  Individual  hostiles  already 
were  modeled  with  reasonable  realism  in  1994,  but  still  a  40%  improvement  in  Current 
Capability  is  indicated  by  1995  survey  responses. 
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Ratings  of  less  than  50%  were  calculated  for  Enemy  prisoners  of  war.  Enemies  in  Allied 
uniforms,  and  Hostile  crowds  —  computer-controlled  entities  for  which  no  TTES  modeling  has 
been  done  so  far. 


Discretionary  Decisions 


Total  Group 


Current  Capability  Rating 


Military 


Individual  hostiles  in  area 


Friendlies/Allied  troops  in  area 


Unknowns  in  area 


Villagers/bystanders  in  area 


Threatening  weapon  presentation 


Fleeing  enemy 


Wounded  enemy 


Snipers  in  area 


Non-threatening  wpn  presentation 


Hostages  in  area 


Enemy  prisoners  of  war 


Enemies  in  Allied  uniforms 


Hostile  crowds 


Current  Overall  TTES  capability: 
Discretionary  Decisions 


1994  Total 
Group 


55. 


41.1 


42.1 


38.9 


52.6 


35.8 


40. 


45. 


43.2 


37.9 


32.9 


35.6 


41.1 


4.  TTES  use:  mission  preview.  Mission  preview  is  a  mission  planning  function  for  a  specific 
area  and  military  operation.  No  dynamic  interactions  with  the  environment  or  potential 
adversaries  are  needed.  It  is  simply  a  virtual  walk-through  of  the  objective  area.  What 
capabilities  should  TTES  provide  for  mission  preview? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 


Potential  Usefulness  Rating 


Mission  Preview 

Total  Group 

Military 

Civilians 

Visualize  potential  kill  zones 

96.4 

96.5 

96.0 

Visualize  obstacles  and  danger  zones 

96.4 

96.5 

96.0 

Visualize  approach  and  routes 

95.7 

95.7 

96.0 

Evaluate  sectors  of  fire  for 
defense/offense 

94.3 

94.8 

92.0 

Familiarize  &  retain  familiarization 

94.3 

93.9 

96.0 

Evaluate  courses  of  action 

94.1 

93.9 

95.0 

Locate  supplementary/alternate 
positions  for  defense/offense 

93.6 

93.0 

96.0 

Solidify  plan  of  action- 

90.7 

91.3 

88.0 

Potential  Overall  TTES  usefulness: 
Mission  Preview 

94.8 

95.0 

93.3 
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•  This  category  was  not  included  in  the  1994  survey,  so  no  Importance  rating  is  available  from 
that  year. 

•  Overall  Potential  Usefulness  and  all  of  the  listed  Mission  Preview  simulation  capabilities  were 
rated  at  better  than  90%  by  the  Total  Group,  and  should  be  considered  critical  to  include  in 
TTES.  The  ability  to  preview  missions  is  expected  to  be  a  very  useful  function  for  TTES. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  The  Mission  Preview  category  of  simulation  factors  was  not  included  in  the  1994  survey,  so  a 
perceived  level  of  improvement  in  the  past  year  cannot  be  determined. 


Current  Capability  Rating 

Mission  Preview 

Total  Group 

Military 

Civilians 

Visualize  approach  and  routes 

76.8 

74.0 

88.0 

Visualize  obstacles  and  danger  zones 

75.8 

76.0 

75.0 

Familiarize  &  retain  familiarization 

74.2 

72.6 

80.0 

Evaluate  courses  of  action 

73.9 

73.0 

80.0 

Visualize  potential  kill  zones 

73.3 

73.0 

75.0 

Locate  supplementary/alternate 
positions  for  defense/offense 

72.7 

72.2 

75.0 

Solidify  plan  of  action 

71.7 

73.0 

65.0 

Evaluate  sectors  of  fire  for 
defehse/offense 

69.6 

67.4 

80.0 

Current  Overall  TTES  capability: 
Mission  Preview 

72.6 

75.0 

60.0 

•  All  factors  in  this  category  received  relatively  high  ratings  from  the  Total  Group,  indicating 
that  TTES  already  is  considered  reasonably  capable  for  previewing  missions. 

4.1.5  Mission  Rehearsal 


5.  TTES  use:  mission  rehearsal.  Mission  rehearsal  is  a  mission  preparation  function  — 
conducting  a  simulated  operation  or  tactical  sequence  in  a  specific  dynamic  environment. 
Hostile  and  neutral  forces  are  tailored  to  the  expected  adversary  profile.  The  goals  are  to 
validate  a  concept  of  operations  and  to  assess  the  scenario  and  potential  outcome.  What 
capabilities  should  TTES  provide  for  mission  rehearsal? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  The  Mission  Rehearsal  category  of  simulation  factors  was  not  included  in  the  1994  survey,  so 
no  Importance  rating  is  available  from  that  year. 

•  All  of  the  factors  included  in  the  Mission  Rehearsal  category  are  considered  critical  to  include 
for  TTES,  with  Overall  and  individual  factor  Potential  Usefulness  ratings  all  over  90%.  As 
with  Mission  Preview,  the  Mission  Rehearsal  function  is  expected  to  be  very  useful  for  trainees 
using  TTES. 
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Mission  Rehearsal 


Potential  Usefulness  Rating 

Total  Group 

Military 

Civilians 

Evaluate  courses  of  action 

96.3 

96.4 

96.0 

React  to  hostile  presence 

95.6 

96.4 

92.0 

Evaluate  plan  of  action 

95.6 

95.5 

96.0 

Cross  danger  point 

94.8 

94.5 

96.0 

Deal  with  threat  point 

94.8 

94.5 

96.0 

Immediate  action  reaction  to  ambush 

93.3 

93.6 

92.0 

React  to  neutral  presence 

93.3 

93.6 

92.0 

Immediate  action  reaction  to  sniper 
fire 

93.3 

92.7 

96.0 

Evaluate  movement  techniques 

93.3 

92.7 

96.0 

Compare  simulation  outcomes  to 
expectation 

92.8 

92.7 

92.0 

Solidify  plan  of  action 

92.6 

91.8 

96.0 

Move  to  supplementary/alternate 
positions 

91.1 

90.0 

96.0 

Potential  Overall  TTES  usefulness: 
Mission  Rehearsal 


Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  The  Mission  Rehearsal  category  of  simulation  factors  was  not  included  in  the  1994  survey,  so 
the  level  of  improvement  in  the  past  year  cannot  be  determined. 


Current  Capability  Rating 


Mission  Rehearsal 


Total  Group  Military 


Civilians 


React  to  hostile  presence 

73.6 

71.4 

85.0 

React  to  neutral  presence 

72.5 

69.0 

90.0 

Evaluate  courses  of  action 

69.0 

67.5 

75.0 

Deal  with  threat  point 

68.6 

65.9 

80.0 

Evaluate  plan  of  action 

67.6 

65.9 

75.0 

Solidify  plan  of  action 

67.6 

65.9 

75.0 

Move  to  supplementary/alternate 
positions 

67.6 

62.4 

90.0 

Compare  simulation  outcomes  to 
expectation 

67.0 

62.5 

85.0 

Evaluate  movement  techniques 

66.7 

62.4 

85.0 

Cross  danger  point 

62.7 

60.0 

75.0 

Immediate  action  reaction  to  ambush 

53.3 

50.0 

65.0 

Immediate  action  reaction  to  sniper 
fire 

50.0 

46.3 

65.0 

Current  Overall  TTES  capability: 
Mission  Rehearsal 


All  factors  in  this  category  received  Total  Group  Current  Capability  ratings  of  50%  or  better, 
indicating  that  TTES  is  considered  at  least  somewhat  capable  now  for  Mission  Rehearsal  — 
about  as  well  developed  as  the  Mission  Preview  category  of  simulation  factors. 


25 


4.2  TTES  OBJECTS,  SITUATIONS,  AND  EVENTS 

Section  B  of  the  1995  survey  concerned  the  details  that  are  needed  for  realistic  modeling  of 
the  training  situations  and  events  included  in  the  first  section’s  five  categories.  Questions  relate  to 
(l)  environments  (urban,  village,  etc.),  (2)  environmental  effects  (weapon  impacts,  fires,  fog,  etc  ), 
and  (3)  general  objects  (animals,  trees,  rocks,  etc.)  that  might  be  included  for  realism.  Questions 
also  were  asked  concerning  (4)  simulated  humans,  (5)  weapons,  and  (6)  sounds.  The  questions 
asked  and  Potential  Usefulness  and  Current  Capability  results  are  provided  below.  In  addition, 
ratings  for  required  Realism  Levels  are  included  for  this  section  of  the  survey. 

4.2.1  Environments 

1 .  Environments  to  simulate  in  TTES.  What  natural  and  man-made  environments  should  be 
simulated  in  the  TTES  synthetic  environment? 

Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  All  but  two  of  the  10  listed  TTES  Environments  are  rated  by  the  Total  Group  at  90%  or  more 
in  Potential  Usefidness,  placing  them  in  the  critical  to  include  category. 


Potential  Usefulness  Rating 

TTES  Environments 

Total  Group 

Military 

Civilians 

Urban:  city 

97.0 

96.4 

100.0 

Jungle 

96.3 

95.5 

100.0 

Urban:  village 

95.6 

94.5 

100.0 

Desert 

95.6 

94.5 

100.0 

Ships,  submarines 

94.1 

93.6 

96.0 

Farmland 

91.5 

90.0 

100.0 

ArcticAvinter 

91.1 

90.9 

92.0 

Federal  buildings 

90.4 

88.2 

100.0 

NBC  environments 

87.2 

85.5 

100.0 

Prisons 

86.7 

84.5 

96.0 

Potential  Overall  TTES 
usefulness:  Environments 

92.0 

90.6 

100.0 

Required  Realism  Level 

73.8 

73.3 

76.0 

•  The  Urban:  city  environment  was  perceived  by  the  Total  Group  as  TTES’s  potentially  most 
useful  environment  (97.0%),  with  Jungle,  Urban:  village,  and  Desert  close  behind. 

•  Inclusion  of  Ships,  submarines  in  the  top  five  factors  is  surprising;  this  factor  rated  last  in 
1994.  No  explanation  is  obvious. 

•  Extremely  high  ratings  were  assigned  by  the  five  civilians  (with  the  Overall  Environments 
category  rated  at  100%).  Law  enforcement  officers  may  anticipate  the  need  for  training  under 
a  very  wide  range  of  conditions. 
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•  The  average  calculated  1994  Importance  rating  for  this  category  was  81.9%.  The  92.0% 
Potential  Usefulness  rating  given  by  the  Total  Group  in  1995  indicates  that  this  latest  group 
of  TTES  evaluators  has  even  stronger  expectations  that  TTES  will  be  useful  for  training  for 
numerous  kinds  of  situations. 

•  The  Total  Group’s  average  rating  for  the  required  Realism  Level  of  simulated  Environments, 
73.8%,  suggests  that  Exact  replication  is  not  necessary;  Good  replication  (hard  to  tell  it  is  not 
real)  should  be  seen  as  adequate  by  potential  TTES  trainees. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  All  of  the  listed  Environments  received  1995  Current  Capability  ratings  of  better  than  50%, 
with  the  Overall  Environments  category  rated  by  the  Total  Group  at  over  74%  of  potential. 


•  The  Current  Capability  ratings  provided  for  all  of  the  individual  Environments  factors  by  the 
1995  survey  respondents  were  surprisingly  high,  considering  that  only  an  Urban:  village 
environment  has  been  implemented  in  the  simulations  so  far.  The  excellence  of  this  village 
simulation  must  have  indicated  to  the  evaluators  that  similarly  good  representations  will  be 
available  for  all  of  the  listed  Environments. 

•  This  Overall  Environments  category  rating  has  jumped  nearly  20  points  in  the  past  year, 
moving  from  a  54.3%  Current  Capability  rating  in  1994  to  74. 1%  in  1995  —  representing  a 
perceived  36%  improvement. 

4.2.2  Environmental  Effects 

2.  Environmental  effects.  What  realistic  environmental  (physical)  occurrences  in  the 

engagement  area  will  influence  TTES  trainees  to  react  tactically  to  the  changing  scenarios? 

Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  All  of  the  listed  Environmental  Effects  were  rated  at  90%  or  better  in  Potential  Usefulness, 
placing  them  all  in  the  critical  classification. 


•  All  five  Civilians  gave  a  rating  of  1  to  all  of  the  listed  effects,  resulting  in  calculated  ratings  of 
100%  for  each.  Law  enforcement  officers  probably  expect  that  they  will  be  required  to  carry 
out  their  duties  under  a  wide  variety  of  conditions  and  circumstances,  during  long  careers,  and 
should  be  trained  for  the  effects  of  all  of  these. 


Potential  Usefulness  Rating 

TTES  Environmental  Effects 

Total  Group 

Military 

Civilians 

Reduced  visibility  (fog,  rain) 

95.8 

95.0 

100.0  j 

Small  arms  projectile  impacts 

95.6 

94.5 

100.0 

Automatic  weapons  projectile  impacts 

95.6 

94.5 

100.0 

Pistol  projectile  impacts 

94.6 

93.3 

100.0 

Grenade  impacts 

92.5 

91.0 

Mortar  impacts 

91.7 

90.0 

100.0 

Shoulder-launched  rocket  impacts 

91.7 

90.0 

100.0 

Structure  fires 

90.4 

88.0 

100.0 

Potential  Overall  TTES  usefulness: 
Environmental  Effects 

92.6 

90.7 

100.0 

Required  Realism  Level 

68.5 

67.6 

72.0 

•  The  1994  calculated  average  Total  Group  Importance  rating  for  the  Environmental  Effects 
category  was  83.7%.  The  related  Potential  Usefulness  rating  of  92.6%  in  1995  represents  a 
10%  increase  in  perceived  usefulness  of  these  factors,  probably  not  a  significant  change  in 
perceptions  considering  the  small  sample  sizes. 

•  The  Total  Group’s  average  rating  for  the  required  Realism  Level,  68.5%,  indicates  that  Good 
replication  (hard  to  tell  it  is  not  real)  is  adequate  for  Environmental  Effects. 


Current  Capability  of  TTES  to  Emulate  Listed  Factors 


Current  Capability  Rating 

TTES  Environmental  Effects 

Total  Group 

Military 

Civilians 

1994  Total 
Group 

Small  arms  projectile  impacts 

57.5 

59.0 

50.0 

43.0 

Automatic  weapons  projectile  impacts 

44.3 

40.0 

55.0 

40.0 

Pistol  projectile  impacts 

43.1 

40.0 

50.0 

37.9 

Structure  fires 

37.8 

37.1 

40.0 

38.9 

Reduced  visibility  (fog,  rain) 

36.0 

37.1 

33.3 

36.7 

Grenade  impacts 

33.3 

33.3 

33.3 

34.4 

Mortar  impacts 

32.0 

31.4 

33.3 

34.4 

Shoulder-launched  rocket  impacts 

32.0 

31.4 

33.3 

35.6 

Current  Overall  TTES  capability: 
Environmental  Effects 

54.1 

58.5 

40.0 

36.0 

•  While  the  Environments  Category  was  rated  relatively  high  in  Current  Capability,  the 
Environmental  Effects  category  was  rated  much  lower.  All  but  one  factor  (Small  arms 
projectile  impacts)  received  calculated  ratings  of  less  than  50%.  More  development  effort 
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probably  should  be  expended  in  this  area,  considering  the  very  high  Potential  Usefulness 
ratings  given  to  this  category. 


•  Structure  fires,  Reduced  visibility,  Grenade  impacts,  Mortar  impacts,  and  Shoulder-launched 
rocket  impacts  (none  of  which  have  been  modeled  for  TTES  at  present)  received  slightly  higher 
Current  Capability  ratings  in  1994  than  in  1995,  though  the  differences  are  not  significant. 

•  Although  still  among  the  lowest-rated  categories  in  Current  Capability,  the  Environmental 
Effects  Total  Group  rating  has  soared  in  the  past  year,  moving  from  36.0%  to  54.1%,  a 
perceived  improvement  of  50%. 

4.2.3  General  Objects 

3.  General  objects  in  TTES  environment.  What  realistic  general  objects  should  be  present  in 
the  TTES  synthetic  environment  to  add  realism  to  training  situations? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  Only  Animals  were  rated  at  less  than  90%  in  Potential  Usefulness  by  the  Total  Group, 

placing  that  factor  in  the  extremely-important- to-include  classification  and  all  the  rest  in  the 
cnrtca/-to-include  classification. 


Potential  Usefulness  Rating 

TTES  General  Objects 

Total  Group 

Military 

Civilians 

Traps,  trip  wires,  mines 

98.6 

98.3 

100.0 

Incoming  fire 

98.6 

98.3 

100.0 

Buildings 

98.6 

98.3 

100.0 

Doors  &  windows 

98.6 

98.3 

100.0 

Roads 

97.9 

97.4 

100.0 

Target  hits 

97.1 

96.5 

100.0 

Furniture 

97.0 

96.4 

100.0 

Barbed  wire/concertina 

96.4 

97.4 

92.0 

Explosions 

96.4 

97.4 

92.0 

Trees,  bushes  (obstacles  &  cover) 

96.4 

95.7 

100.0 

Rocks  (obstacles  &  cover) 

96.4 

95.7 

100.0 

Vehicles 

96.4 

95.7 

100.0 

Burning  objects 

95.6 

95.5 

96.0 

Debris 

95.6 

94.5 

100.0 

Rivers,  lakes,  etc. 

95.0 

94.8 

96.0 

Bridges 

95.0 

94.8 

96.0  1 

Rubble 

93.3 

93.6 

92.0 

Animals 

80.7 

77.4 

96.0 

Potential  Overall  TTES  usefulness: 
General  Objects 

96.8 

97.3 

95.0 

Required  Realism  Level 

79.3 

78.2 

84.0 

•  In  1994,  General  Objects  received  a  calculated  Total  Group  average  Importance  rating  of 
84.9%,  compared  with  a  96.8%  Overall  Potential  Usefulness  rating  in  1995,  about  a  15% 
increase. 
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•  The  Total  Group’s  average  rating  for  the  required  Realism  Level  of  General  Objects,  79.3%,  is 
the  highest  for  any  category,  barely  below  the  arbitrary  cutoff  for  Exact  replication. 

Obviously,  very  good  emulation  is  desired  for  simulated  objects. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Eight  of  the  listed  General  Objects  were  rated  at  better  than  50%  in  Current  Capability  by  the 
Total  Group. 


Current  Capability  Rating 

TTES  General  Objects 

Total  Group 

Military 

Civilians 

1994  Total 
Group 

Buildings 

82.4 

81.8 

86.7 

63.0 

Roads 

78.2 

78.9 

73.3 

60.0 

Doors  &  windows 

74.8 

74.0 

80.0 

62.0 

Trees,  bushes  (obstacles  &  cover) 

71.4 

71.1 

73.3 

48.0 

Target  hits 

62.4 

61.5 

65.0 

49.0 

Rocks  (obstacles  &  cover) 

60.0 

58.0 

70.0 

38.9 

Rivers,  lakes,  etc. 

52.5 

46.7 

70.0 

43.3 

Bridges 

52.5 

46.7 

70.0 

46.7 

Incoming  fire 

48.3 

49.1 

40.0 

36.8 

Explosions 

43.3 

44.0 

40.0 

38.9 

Burning  objects 

43.3 

44.0 

40.0 

41.1 

Vehicles 

42.9 

44.0 

40.0 

58.0 

Animals 

40.0 

40.0 

40.0 

36.7 

Barbed  wire/concertina 

40.0 

40.0 

40.0  | 

38.9 

Debris 

40.0 

40.0 

40.0 

36.7 

Furniture 

38.2 

40.0 

30.0 

40.0 

Rubble 

37.1 

36.7 

40.0 

43.2 

Traps,  tripwires,  mines 

36.7 

36.0 

40.0 

38.9 

Current  Overall  TTES 
capability:  General  Objects 

64.4 

66.7 

53.3 

42.9 

•  The  representation  of  Buildings  was  top  rated  at  82.4%  —  the  highest  Total  Group  Current 
Capability  rating  for  any  of  the  individual  factors  included  in  any  simulation  category.  TTES 
developers  obviously  are  perceived  as  having  done  an  admirable  job  of  emulating  realistic 
buildings. 

•  Most  of  the  listed  General  Objects  still  need  improvement  in  simulation  fidelity,  with  10  of  the 
18  rated  below  50%  in  Current  Capability  by  the  Total  Group. 

•  The  Total  Group’s  Overall  rating  for  the  simulation  of  General  Objects  has  gone  from  42.9% 
to  64.4%  in  the  past  year,  an  increase  of  50%. 

4.2.4  Simulated  Humans 


4.  Simulated  humans  in  TTES  environment.  What  humans  should  be  present  in  the  TTES 
synthetic  environment  for  trainees  to  interact  with,  as  they  go  through  training  situations? 
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Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  The  Simulated  Humans  category  of  simulation  factors  was  rated  very  high  in  Potential 
Usefulness  by  the  Total  Group,  at  96.4%. 


Potential  Usefulness  Rating 

TTES  Simulated  Humans 

Total  Group 

Military 

Civilians 

Hostiles 

97.9 

97.4 

100.0 

Snipers 

97.1 

96.5 

100.0 

Friendlies 

95.7 

94.8 

100.0 

Villagers/innocent  bystanders 

95.0 

93.9 

100.0 

Hostages 

95.0 

93.9 

100.0 

Unknowns 

94.3 

93.0 

100.0 

Potential  Overall  TTES  usefulness: 
Simulated  Humans 

96.4 

95.6 

100.0 

Required  Realism  Level 

74.8 

73.6 

80.0 

•  Although  Hostiles  and  Snipers  ranked  at  the  top  of  the  Potential  Usefulness  list,  differences 
between  the  ratings  of  these  two  and  those  of  the  remaining  four  listed  types  of  humans  are  not 
significant. 

•  Each  of  the  five  Civilians  gave  a  rating  of  1  to  all  five  types  of  listed  Simulated  Humans, 
resulting  in  calculated  100%  ratings  across  the  board. 

•  The  Simulated  Humans  category  ranked  highest  in  the  1994  survey,  with  a  calculated  average 
Importance  rating  of  92.7%.  The  1995  96.4%  Overall  Potential  Usefulness  rating,  while 
higher,  does  not  represent  a  significant  difference  (only  4%)  —  but  probably  indicates  that,  as 
modeling  has  improved,  TTES  users  still  feel  that  including  a  reasonable  range  of  computer- 
controlled  battlefield  entities  is  very  important. 

•  A  Total  Group  required  Realism  Level  rating  of  74.8%  for  Simulated  Humans  indicates  that 
Good  replication  (hard  to  tell  it  is  not  real)  should  be  seen  as  adequate  by  potential  TTES 
trainees  The  rating  by  Civilians  is  on  the  border  of  suggesting  that,  for  law  enforcement. 
Exact  emulation  of  humans  may  be  desired. 


Current  Capability  of  TTES  to  Emulate  Listed  Factors 


Current  Capability  Rating 

TTES  Simulated  Humans 

Total  Group 

Military 

Civilians 

1994  Total 
Group 

Hostiles 

76.3 

73.9 

90.0 

58.9 

Friendlies 

68.7 

64.2 

90.0 

44.4 

Villagers/innocent  bystanders 

67.6 

62.4 

90.0 

44.4 

Unknowns 

59.0 

51.3 

90.0 

43.3 

Snipers 

53.8 

41.7 

90.0 

48.9 

Hostages 

46.2 

41.8 

70.0 

i - 1 

50.0 

Current  Overall  TTES  capability: 
Simulated  Humans  _ 


69.5 


67.8 


80.0 


42.9 


•  With  the  exception  of  Hostages  (46.2%),  the  TTES  evaluators  rated  all  of  the  Simulated 
Humans  at  better  than  50%  in  Current  Capability. 

•  Hostiles  still  are  considered  the  best  modeled  humans  in  1995,  as  in  1994.  However,  based  on 
the  ratings  from  the  two  years,  about  a  30%  improvement  is  perceived  as  a  result  of  the  past 
year’s  development  efforts. 

•  Modeling  of  Friendlies  (the  other  member  of  the  two-man  squad  carrying  out  each  scenario) 
and  Villagers  is  new  in  1995.  Thus  it  is  not  surprising  that  the  ratings  indicate  major 
improvements  in  simulation  of  these  humans:  more  than  50%  for  each. 

•  The  group  of  five  Civilians  was  especially  positive  about  current  modeling  of  Simulated 
Humans  in  TTES,  with  all  humans  except  Hostages  rated  at  90%  and  an  Overall  rating  of 
80%  for  the  category. 

•  Overall,  modeling  of  Simulated  Humans  is  rated  more  than  25  points  higher  in  1995  than  in 
1994,  a  perceived  improvement  of  over  60%. 

4.2.5  Weapons 


5.  Weapons  in  TTES  environment.  What  weapon  systems  should  be  present  in  the  TTES 
synthetic  environment  for  trainees  to  practice  using  and  to  interact  with? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 


Potential  Usefulness  Rating 

TTES  Weapons 

Total  Group 

Military 

Civilians 

Assault  rifles 

97.9 

97.4 

100.0 

Squad  automatic  weapons 

97.8 

97.4 

100.0 

Pistols 

97.1 

96.5 

100.0 

Submachine  guns 

97.0 

96.4 

100.0 

Machine  guns 

96.2 

95.5 

100.0 

Shoulder-launched  missiles/rockets 

91.9 

91.3 

95.0 

Grenades 

89.6 

87.8 

100.0 

Flashbangs 

87.1 

84.3 

100.0 

Satchel  charges 

86.9 

84.5 

100.0 

Mortars 

85.4 

83.6 

95.0 

Explosive  charges 

83.7 

81.7 

95.0 

Knives 

75.4 

73.6 

85.0 

Potential  Overall  TTES  usefulness: 
Weapons 

91.0 

90.6 

93.3 

Required  Realism  Level 

78.5 

76.4 

88.0 

•  Total  Group  ratings  of  more  than  90%  were  calculated  for  the  Potential  Usefulness  of  six 
Weapons:  Assault  rifles.  Squad  automatic  weapons,  Pistols,  Submachine  guns,  Machine  guns, 
and  Shoulder-launched  missiles/rockets.  Modeling  of  these  falls  into  the  critical  classification. 
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Grenades,  while  technically  below  the  90%  cutoff  with  89.6%,  probably  also  belong  in  that 
classification. 

•  Knives,  rated  at  75.4%  by  the  Total  Group,  are  in  the  important- to-include  classification, 
while  the  four  remaining  listed  Weapons  (ratings  between  80  and  90%)  should  be  considered 
extremely  important. 

•  The  five  Civilians  generally  considered  all  of  the  listed  weapons  more  Potentially  Useful  than 
did  the  Military  respondents,  but  they,  too,  rated  knives  at  the  bottom. 

•  Weapons  received  an  overall  81.8%  calculated  Importance  rating  in  1994  and  a  91.0% 
Potential  Usefulness  rating  in  1995,  indicating  about  a  10%  difference  in  the  perceived 
importance  of  including  a  variety  of  Weapons  in  TTES. 

•  Total  Group  average  rating  for  the  required  Realism  Level  of  weapons,  78.5%,  is  the  second 
highest  (after  General  Objects)  among  all  Section  B  categories.  This  indicates  that,  while 
Exact  replication  may  not  be  necessary;  Good  replication  (hard  to  tell  it  is  not  real)  is  very 
important  for  this  category.  The  group  of  five  Civilians  rated  weapons  as  needing  the  greatest 
realism  of  any  category,  and  propose  that  they  require  Exact  replication. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Modeling  of  only  the  Assault  rile  approaches  a  reasonable  level  of  Current  Capability 
(78.4%).  This  weapon  is  perceived  as  being  about  30%  more  realistic  in  1995  than  it  was  in 
1994. 


Current  Capability  Rating 

TTES  Weapons 

Total  Group 

Military 

Civilians 

Assault  rifles 

78.4 

76.2 

90.0 

58.9 

Submachine  guns 

42.2 

34.3 

70.0 

35.3 

Machine  guns 

40.0 

42.5 

30.0 

38.9 

Knives 

37.8 

28.6 

70.0 

31.1 

Pistols 

34.0 

35.0 

30.0 

38.8 

Explosive  charges 

32.5 

31.4 

40.0 

34.1 

Flashbangs 

32.5 

31.4 

40.0 

33.3 

Squad  automatic  weapons 

32.5 

31.4 

40.0 

37.8 

Shoulder-launched  missiles/rockets 

32.5 

31.4 

40.0 

34.4 

Mortars 

30.0 

28.6 

40.0 

32.9 

Grenades 

30.0 

28.6 

40.0 

34.4 

Satchel  charges 

30.0 

28.6 

40.0 

33.3 

Current  Overall  TTES  capability: 
Weapons 

63.2 

67.1 

30.0 

33.3 

•  The  remaining  weapons  (none  of  which  are  available  in  TTES  at  present)  are  rated  very  low  in 
Current  Capability,  as  they  were  in  1994. 

•  The  five  Civilians  rated  the  Current  Capability  of  nearly  all  of  the  weapons  higher  than  did  the 
Military  group  but,  surprisingly,  gave  a  much  lower  Overall  rating  for  the  category. 
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•  In  spite  of  the  low  ratings  assigned  to  most  of  the  listed  weapons,  the  Total  Group’s  Overall 
Weapons  Current  Capability  rating  of  63.2%  is  quite  respectable,  especially  w'hen  compared 
with  the  33.3%  value  assigned  in  1994  (a  90%  perceived  improvement  over  the  year). 

4.2.6  Sounds 


6.  Sounds  in  TTES  environment.  What  audible  sounds  should  be  present  in  the  TTES 
synthetic  environment  to  add  realism  to  training  situations'? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  This  is  the  first  year  when  any  battlefield  Sounds  were  emulated  in  TTES  demonstrations. 
Verbal  communications  between  the  two  members  of  the  squad  were  implemented  in  1995  via 
microphones  and  audio  headsets. 


Potential  Usefulness  Rating 

TTES  Sounds 

Total  Group 

Military 

Civilians 

Normal  conversations 

96.3 

96.4 

96.0 

Verbal  commands 

95.6 

97.3 

88.0 

Explosions 

95.0 

93.9 

100.0 

Excited  chatter 

94.8 

93.9 

100.0 

Weapon  functioning 

94.3 

93.0 

100.0 

Footsteps 

91.9 

90.9 

96.0 

Trees  &  branches  snapping 

88.1 

86.4 

96.0 

Potential  Overall  TTES 
usefulness:  Sounds 

98.0 

98.8 

95.0 

Required  Realism  Level 

76.9 

76.2 

80.0 

•  With  the  exception  of  Trees  &  branches  snapping  (88. 1%  and  considered  extremely  important 
to  include),  all  of  the  listed  Sounds  are  in  the  critical  classification  with  Total  Group  ratings 
over  90%  in  Potential  Usefulness. 

•  Sounds,  with  a  calculated  average  rating  of  86  7%  in  1994,  at  that  time  ranked  second  in 
Importance  only  to  Simulated  Humans.  This  year’s  rating  of  98.0%  moves  the  Sounds 
category  ahead  of  the  Simulated  Humans  category  (96.4%)  in  Potential  Usefulness  for  TTES 
training. 

•  The  Total  Group’s  average  rating  for  required  Realism  Level,  76.9%,  suggests  that  Exact 
replication  of  Sounds  is  not  quite  necessary;  Good  replication  (hard  to  tell  it  is  not  real) 
probably  will  be  seen  as  adequate  by  most  TTES  trainees. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Even  though  the  only  Sounds  implemented  so  far  are  verbal  communications,  all  of  the  listed 
types  were  rated  much  higher  in  Current  Capability  in  1995  than  in  1994.  Apparently  the 
ability  to  communicate  during  training  exercises  has  indicated  to  TTES  evaluators  that  other 
Sounds  now  also  are  possible  or  will  be  soon. 
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Current  Capability  Rating 

TTES  Sounds 

Total  Group 

Military 

Civilians 

1994  Total 
Group 

Normal  conversations 

66.3 

67.5 

60.0 

32.2 

Verbal  commands 

58.6 

58.2 

60.0 

32.2 

Weapon  functioning 

54.7 

51.4 

64.0 

35.6 

Footsteps 

50.7 

48.3 

60.0 

32.2 

Explosions 

45.0 

40.0 

60.0 

33.3 

Excited  chatter 

41.3 

36.4 

55.0 

31.1 

Trees  &  branches  snapping 

36.4 

27.5 

60.0 

33.3 

Current  Overall  TTES  capability: 
Sounds 

60.0 

65.3 

33.3 

30.7 

•  Normal  conversations.  Verbal  commands.  Weapon  functioning,  and  Footsteps  received 
Current  Capability  ratings  of  50%  or  better. 

•  The  remaining  three  Sounds  have  not  been  implemented  in  TTES  and  are  rated  much  lower. 

•  The  five  Civilians  gave  reasonably  high  Current  Capability  ratings  to  the  individual  listed 
Sounds  (55  to  64%),  but  only  a  33.3%  Overall  rating  for  the  category  as  a  whole. 

•  As  noted,  1995  was  the  first  year  when  any  battlefield  Sounds  were  simulated.  Thus  a 
perceived  Current  Capability  improvement  of  95%  from  the  1994  ratings  is  not  surprising. 

•  The  great  importance  the  subject  matter  experts  have  assigned  to  Sounds  and  the  category’s 
relatively  low  Current  Capability  rating  indicate  that  this  is  an  area  that  TTES  developers 
should  stress. 

4.3  TRAINEE  AND  ADVERSARY  MOVEMENTS  AND  POSITIONS 

Section  C  of  the  1995  survey  concerns  the  level  of  detail  needed  for  both  realistic  instruction 
of  trainees  and  realistic  emulation  of  simulated  adversaries  in  tactical  situations.  Questions  were 
asked  related  to  (1)  coordinated  tactical  movements,  (2)  individual  body  movements,  (3)  firing 
positions,  (4)  arm  and  hand  movements,  (5)  head  movements,  and  (6)  eye  movements. 

For  each  question,  survey  participants  were  asked  to  provide  two  Potential  Usefulness 
ratings:  one  for  how  useful  it  will  be  that  the  Trainee  have  that  capability,  and  a  second  for  how 
useful  it  will  be  that  Adversaries  are  simulated  at  this  level  of  detail.  Respondents  also  were  asked 
for  two  Current  Capability  ratings,  for  both  Trainee  and  Adversary  capabilities. 

For  every  category.  Overall  Potential  Usefulness  ratings  for  the  listed  factors  were  higher  for 
Trainees  than  for  simulated  Adversaries.  The  experts  who  participated  in  this  survey  apparently 
feel  that  some  deviation  from  real-world  behavior  by  simulated  humans  may  be  acceptable,  but  that 
Trainees  must  be  able  to  carry  out  typical  combat  tasks  as  realistically  as  possible  if  TTES  is  to  be 
accepted. 
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4.3.1  Coordinated  Tactics 


l.a.  Trainee  coordinated  tactical  movements.  What  coordinated  tactical  movements  should 
TTES  allow  trainees  to  perform  in  the  simulator,  as  they  go  through  simulated  training 
situations? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  The  survey  respondents’  Potential  Usefulness  ratings  indicated  that  all  but  one  of  the  listed 
Coordinated  Tactical  Movements  are  critical  for  Trainees. 


Potential  Usefulness  Rating  for  Trainee 

Coordinated  Tactical  Movements 

Total  Group 

Military 

Civilian 

Enter  room 

99.3 

99.1 

100.0 

Enter  building 

98.6 

98.3 

100.0 

Patrol 

98.5 

98.2 

100.0 

Enter  ambush  site 

97.0 

96.4 

100.0 

Fire  &  maneuver 

96.4 

95.7 

100.0 

Rush  (fire  &  move) 

96.2 

95.2 

100.0 

Cover  moving  buddies 

94.3 

96.5 

84.0 

Use  hand  &  arm  signals 

93.6 

93.0 

96.0 

Enter  vehicle 

91.2 

89.5 

100.0 

Control  crowds/marshal  people 

90.0 

88.2 

100.0 

Pick  up/carry  injured  buddy 

80.8 

78.1 

95.0 

Potential  Overall  TTES 
usefulness:  Tactical  Movements 

96.2 

95.3 

100.0 

•  Only  Pick/carry  injured  buddy  was  rated  less  than  90%  in  Potential  Usefulness  for  the 
Trainee,  and  even  that  capability  fell  into  the  extreme ly-important-to-inc\ude  classification. 


•  Coordinated  Tactics  were  considered  quite  important  by  the  five-man  Civilian  subgroup, 
which  gave  a  100%  Overall  Potential  Usefulness  rating  to  this  category  for  Trainees. 

•  In  1994,  this  category  received  an  Overall  Importance  rating  of  85.2%.  The  Potential 
Usefulness  jump  to  96.2%  in  1995  represents  a  perceived  increase  in  importance/usefulness 
of  about  12%. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Enter  buildings.  Enter  rooms.  Patrol,  and  Cover  moving  buddies  all  were  rated  relatively 
highly  (60  to  70%)  in  Current  Capability  in  1995.  Fire  &  maneuver  also  was  rated  at  better 
than  50%. 


•  The  six  remaining  Coordinated  Tactical  Movements  listed  in  this  category  were  rated  below 
50%,  with  four  (Pickup/carry  injured  buddy,  Use  hand  &  arm  signals,  Enter  vehicle,  and 
Control  crowds/marshal  people)  rated  very  low  in  Current  Capability  (among  the  lowest  of 
any  factors  included  on  the  questionnaire).  This  is  not  surprising,  since  none  of  these 
currently  have  been  implemented  in  TTES,  and  it  is  hard  even  to  envision  how  they  might  be 
included  in  TTES  scenarios. 
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Current  Capability  Rating  for  Trainee 

Coordinated  Tactical  Movements 

Total  Group 

Military 

Civilian 

1994  Total 
Group 

Enter  building 

70.0 

72.4 

60.0 

57.3 

Enter  room 

68.9 

70.9 

60.0 

57.3 

Patrol 

61.1 

58.6 

Too 

45.3 

Cover  moving  buddies 

61.0 

57.5 

75.0 

31.4 

Fire  &  maneuver 

52.7 

52.9 

52.0 

38.7 

Rush  (fire  &  move) 

49.0 

48.0 

52.0 

37.1 

Enter  ambush  site 

44.0 

38.2 

60.0 

41.3 

Pick  up/carry  injured  buddy 

29.3 

30.0 

26.7 

28.6 

Use  hand  &  arm  signals 

28.2 

27.1 

33.3 

32.9 

Enter  vehicle 

26.2 

26.0 

26.7 

32.9 

Control  crowds/marshal  people 

26.2 

26.0 

26.7 

30.0 

Current  Overall  TTES  capability: 
Tactical  Movements 

mm 

60.0 

50.0 

mm 

•  In  spite  of  the  preponderance  of  poor  Current  Capability  ratings  for  Coordinated  Tactical 
Movements,  significant  improvement  in  the  last  year  is  indicated  by  comparing  the  1994  and 
1995  results;  there  is  about  a  45%  difference  in  the  two  years’  Overall  ratings. 


l  b.  Simulated  adversary  coordinated  tactical  movements.  What  coordinated  tactical 

movements  should  TTES  simulate  for  hostiles,  neutrals,  and  friendlies,  for  various  training 
situations? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 


Potential  Usefulness  Rating  for  Adversary 


Coordinated  Tactical  Movements  Total  Group 


Enter  room 


Enter  buildin 


Enter  ambush  site 


Rush  (fire  &  move) _ 


Fire  &  maneuver 


Patrol  _ 


Cover  moving  buddies _ 


Use  hand  &  arm  signals 


Enter  vehicle  _ 


Pick  up/carry  injured  buddy _ 


Control  crowds/marshal  people _ _ 


Potential  Overall  TTES  usefulness: 
Tactical  Movements 


•  Computer-controlled  Adversaries  are  expected  by  the  Total  Group  to  display  nearly  the  same 
level  of  Coordinated  Tactical  Movements  as  Trainees  are  permitted  in  TTES:  94.7%  Overall 
Potential  Usefulness  to  the  Trainee’s  96.2%. 

•  As  with  Trainees,  Pickup/carry  injured  buddy  and  Control  crowds/marshal  people  were  rated 
lowest  in  Potential  Usefulness  for  Adversaries,  though  still  at  87.0  and  82.5%,  respectively. 

•  Civilians  consistently  rated  the  Potential  Usefulness  of  the  listed  Tactical  Movements  as 
lower  for  Adversaries  than  for  Trainees. 

•  In  1994  this  category  received  an  Importance  rating  of  only  74.6%,  raising  concerns  that 
military  personnel  may  be  underestimating  the  capabilities  of  their  real-world  adversaries  and 
thus  proposing  that  their  actions  only  need  be  modeled  in  a  simplistic  manner.  The  1995 
Overall  Coordinated  Tactical  Movements  Potential  Usefulness  rating  for  Adversaries  of 
94.7%  (more  than  a  25%  increase)  alleviates  these  concerns. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  TTES  Adversaries  are  seen  by  the  Total  Group  as  more  than  50%  capable  for  only  three 
types  of  Tactical  Movements:  Patrol,  Enter  room,  and  Enter  building. 


Current  Capability  Rating  for  Adversary 

Coordinated  Tactical  Movements 

Total  Group 

Military 

Civilian 

1994  Total 
Group 

Patrol 

60.0 

58.6 

70.0 

45.3 

Enter  room 

57.9 

60.0 

46.7 

57.3 

Enter  building 

56.8 

58.8 

46.7 

|  57.3 

Rush  (fire  &  move) 

44.0 

44.6 

40.0 

37.1 

Cover  moving  buddies 

43.8 

36.9 

73.3 

31.4 

Enter  ambush  site 

43.1 

40.0 

60.0 

41.3 

Fire  &  maneuver 

42.7 

43.1 

40.0 

38.7 

Pick  up/carry  injured  buddy 

34.5 

35.6 

30.0 

28.6 

Enter  vehicle 

33.3 

34.0 

30.0 

32.9 

Use  hand  &  arm  signals 

31.4 

30.0 

40.0 

32.9 

Control  crowds/marshal  people 

30.0 

30.0 

30.0 

30.0 

Current  Overall  TTES  capability: 
Tactical  Movements 

47.1 

49.1 

40.0 

■a 

•  Current  Capability  ratings  of  less  than  50%  were  calculated  for  the  remaining  eight  of  the  11 
listed  movements.  The  same  movements  considered  unsatisfactory  for  Trainees  also  generally 
are  judged  as  unsatisfactory  for  the  simulated  Adversaries. 

•  In  general,  the  individual  ratings  indicate  that  little  progress  has  been  made  in  simulation  of 
these  Coordinated  Tactical  Movements  for  Adversaries  over  the  past  year  —  with  the 
exception  of  the  ability  to  Patrol. 
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•  While  about  an  1 8%  improvement  can  be  inferred  from  the  1995  Overall  category  ratings 
when  compared  with  1994,  the  47. 1%  Current  Capability  value  calculated  for  Adversary 
Coordinated  Tactical  Movements  indicates  that  major  improvements  in  realism  are  needed. 
This  category  ranked  lowest  of  all  in  perceived  capability. 

4.3.2  Body  Movements 

2.a.  Individual  trainee  body  movements.  What  tactically  correct  whole-body  movements  should 
TTES  allow  trainees  to  perform  in  the  simulator? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  All  listed  Individual  Body  Movements  were  rated  at  better  than  90%  in  Potential  Usefulness 
for  Trainees.  Each  of  these  capabilities  must  be  considered  critical  to  include  in  TTES  for 
Trainees  to  use  as  appropriate  in  training  scenarios. 


Potential  Usefulness  Rating  for  Trainee 

Individual  Body  Movements 

Total  Group 

Military 

Civilian 

Run 

97.8 

99.1 

92.0 

Walk  forwards 

97.1 

98.3 

92.0 

Walk  backwards 

97.1 

98.3 

92.0 

Stand  up 

97.0 

98.2 

92.0 

Lean  over 

96.9 

98.1 

92.0 

Duck 

96.9 

98.1 

92.0 

Jump 

96.9 

98.1 

92.0 

Climb/crawl  over  obstacles 

96.9 

98.1 

92.0 

Dive  behind  cover 

96.8 

98.1 

90.0 

Kneel  down 

96.4 

97.4 

92.0 

Lie  down  on  front 

96.4 

97.4 

92.0 

Rotate  body 

96.4 

97.4 

92.0 

Crawl 

96.3 

97.4 

90.0 

Roll  right  &  left 

95.4 

96.2 

92.0 

Hit  &  roll 

94.6 

95.2 

92.0 

Lie  down  on  back 

93.1 

93.3 

92.0 

Squat/duck  walk 

92.3 

92.4 

92.0 

Potential  Overall  TTES 
usefulness:  Body  Movements 

95.2 

96.5 

90.0 

•  An  Importance  rating  of  88.0  was  calculated  for  this  category  in  1994.  While  the  95.2% 
rating  calculated  for  Potential  Usejulness  in  1995  is  higher,  it  represents  less  than  10% 
change  in  perceived  importance/usefulness  over  the  past  year. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  The  Total  Group  of  TTES  evaluators  gave  Current  Capability  ratings  of  better  than  50%  to 
10  of  the  17  listed  Individual  Body  Movements  for  Trainees,  with  all  of  these  10  apparently 
much  improved  over  the  past  year. 


39 


Current  Capability  Rating  for  Trainee 


_ 

Individual  Body  Movements 

Total  Group 

Military 

Civilian 

1994  Total 
Group 

Stand  up 

77.5 

78.0 

75.0 

62.5 

Knee!  down 

75.4 

74.5 

80.0 

58.8 

Walk  backwards 

69.6 

70.5 

65.0 

56.3 

Run 

67.3 

71.1 

50.0 

47.5 

Rotate  body 

67.3 

67.8 

65.0 

43.8 

Walk  forwards 

67.2 

67.6 

65.0 

53.8 

Squat/duck  walk 

65.3 

65.0 

66.7 

41.3 

Lie  down  on  front 

61.9 

58.8 

75.0 

55.0  j 

Duck 

61.1 

61.3 

60.0 

52.5 

Lean  over 

61.1 

57.1 

75.0 

43.8 

Crawl 

47.8 

48.0 

46.7 

33.3 

Roil  right  &  left 

43.8 

40.0 

60.0 

37.3 

Lie  down  on  back 

38.7 

33.3 

60.0 

40.0 

Hit  &  roll 

37.3 

31.7 

60.0 

33.3 

Jump 

32.5 

26.7 

50.0 

33.3 

Dive  behind  cover 

31.4 

27.3 

46.7 

33.3 

Climb/crawl  over  obstacles 

29.2 

30.0 

26.7 

32.9 

Current  Overall  TTES  capability: 
Body  Movements 

60.0 

61.3 

53.3 

44.6 

•  Most  of  the  remaining  seven  Body  Movements,  rated  between  47.8  and  29.2%  in  1995, 
showed  little  or  no  improvement  in  the  past  year. 

•  The  Overall  rating  for  Trainee  Body  Movements,  however,  was  significantly  greater  in  1995 
than  in  1994,  indicating  more  than  30%  improvement  in  Current  Capability.  Thus,  while 
potential  TTES  users  still  are  dissatisfied  with  their  ability  to  move  freely  in  TTES,  they  are 
less  unhappy  than  they  were  a  year  ago. 


2.b.  Simulated  adversary  movements.  What  whole-body  movements  should  TTES  simulate  for 
hostiles,  neutrals,  and  friendlies,  as  trainees  encounter  them  during  training  situations? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  While  a  slightly  lower  Overall  rating  is  given  by  the  Total  Group  for  the  Potential  Usefulness 
of  these  Body  Movements  for  the  Adversary  than  for  Trainees,  in  general  the  evaluators 
indicated  that  they  expect  the  entities  they  encounter  in  TTES  to  display  the  same  human 
capabilities  as  they  themselves  have  —  that  is,  the  computer-controlled  simulated  humans 
should  be  as  “real”  as  possible. 
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Potential  Usefulness  Rating  for  Adversary 


Individual  Body  Movements 

Total  Group 

Military 

Civilian 

Stand  up 

96.4 

98.3 

88.0 

Walk  forwards 

96.4 

98.3 

88.0 

Walk  backwards 

96.4 

98.3 

88.0 

Kneel  down 

96.3 

98.2 

*  88.0 

Run 

96.3 

98.2 

88.0 

Crawl 

95.7 

97.4 

88.0 

Climb/crawl  over  obstacles 

95.4 

97.1 

88.0 

Lie  down  on  front 

95.0 

96.5 

88.0 

Roll  right  &  left 

94.8 

96.4 

88.0 

Lean  over 

94.8 

96.4 

88.0 

Duck 

94.8 

96.4 

88.0 

Dive  behind  cover 

94.6 

96.4 

85.0 

Jump 

93.6 

95.2 

85.0 

Squat/duck  walk 

93.6 

94.8 

88.0 

Rotate  body 

93.6 

94.8 

88.0 

Hit  &  roll 

93.3 

94.5 

88.0 

Lie  down  on  back 

91.4 

92.2 

88.0 

Potential  Overall  TTES 
usefulness:  Body  Movements 

94.3 

96.5 

85.0 

•  The  1994  survey  resulted  in  a  83.5%  Importance  rating  for  this  category,  while  94.3%  was 
calculated  for  Potential  Usefulness  in  1995  (about  a  12%  difference).  The  experts  surveyed 
this  year  apparently  consider  their  real-world  Adversaries  to  be  about  as  capable  as 
themselves,  and  want  them  modeled  that  way. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Eight  of  the  17  Individual  Body  Movements  were  rated  by  the  Total  Group  above  50%  in 
Current  Capability  for  Adversaries;  reasonable  progress  is  perceived  for  the  last  year  in 
modeling  these  movements. 

•  Ratings  for  the  remaining  nine  Body  Movements  ranged  from  47.5  down  to  32.9%.  These 
areas  are  seen  as  needing  additional  enhancements  in  capability,  with  little  or  no  progress 
made  in  the  past  year  (in  fact,  Kneel  down  was  rated  somewhat  worse  this  year). 


•  Overall,  Adversaries’  Current  Capability  to  carry  out  complex  Body  Movements  was  rated 
by  the  Total  Group  at  only  58.9%  in  1995.  However,  this  compares  with  44.6%  in  1994  — 
indicating  about  a  30%  improvement.  Progress  is  being  made! 
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3. a.  Trainee  firing  positions.  What  tactically  correct  firing  positions  and  other  body  positions 
should  TTES  allow  trainees  to  use  in  the  simulator? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 


Firing  Positions 


Kneeling 


Standing 


From  holes 


Through  doors 


Through  windows 


Squatting  or  crouched 


From  behind  trees 


From  under  vehicles 


Prone 


Barricaded 


Supported 


Sitting 


On  back 


Potential  Usefulness  Rating  for  Trainee 

Total  Group 

Military 

Civilian 

Potential  Overall  TTES 
usefulness:  Firing  Positions 
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•  With  the  exception  of  On  back  (which  rated  fairly  low),  the  TTES  evaluators  consider  all  of 
the  listed  Firing  Positions  to  have  very  high  Potential  Usefulness  (better  than  90%),  placing 
them  in  the  critical  classification  for  Trainees. 

•  Calculated  ratings  for  the  Civilian  group  were  100%  for  all  of  the  listed  Body  Movements. 

•  An  average  Total  Group  Importance  rating  of  88.2%  was  calculated  for  this  category  in 
1994.  This  is  less  than  10%  lower  than  the  96.0%  Potential  Usefulness  rating  given  in  1995, 
indicating  that  the  1994  and  1995  groups  held  similar  opinions  about  the  importance/ 
usefulness  of  these  Firing  Positions  for  Trainees. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors- 

•  The  seven  Firing  Positions  that  were  rated  better  than  50%  in  Current  Capability  for  Trainees 
generally  reflect  the  capabilities  currently  implemented  in  TTES.  The  ratings  range  from  80.7 
to  5 1.3%,  indicating  that  standing  is  considered  to  be  quite  well  implemented,  while  sitting 
would  be  harder  to  do  in  the  current  version  of  TTES. 


Current  Capability  Rating  for  Trainee 

Firing  Positions 

Total  Group 

Military 

Civilian 

1994  Total 
Group 

Standing 

80.7 

80.0 

84.0 

64.7 

Kneeling 

73.6 

71.4 

85.0 

54.1 

Through  windows 

72.0 

71.3 

75.0 

57.6 

Through  doors 

72.0 

70.7 

76.0 

58.8 

Squatting  or  crouched 

67.0 

61.3 

84.0 

55.3 

Prone 

64.2 

60.0 

80.0 

51.8 

Sitting 

51.3 

44.6 

80.0 

51.8 

Supported 

48.0 

41.7 

73.3 

36.3 

From  holes 

47.1 

41.8 

66.7 

44.7 

Barricaded 

45.3 

36.7 

80.0 

33.8 

From  behind  trees 

40.0 

35.0 

60.0 

33.8 

From  under  vehicles 

40.0 

34.0 

60.0 

46.3 

On  back 

37.1 

27.3 

73.3 

o 

© 

Tf 

Current  Overall  TTES  capability: 
Firing  Positions 

57.3 

56.7 

60.0 

46.2 

•  The  remaining  six  Firing  Positions  are  not  possible  in  TTES  systems  at  present,  so  low 
satisfaction  with  Current  Capability  is  not  surprising. 


•  All  but  three  of  the  listed  Firing  Positions  are  perceived  as  being  improved  since  last  year, 
though  the  differences  mostly  are  not  important. 

•  The  1995  Overall  Current  Capability  rating  for  Trainee  Firing  Positions,  57.3%,  compares 
with  a  value  of  46.2%  for  1994,  suggesting  that  significant  improvements  may  have  been 
made  in  the  past  year. 
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3.b.  Simulated  adversary  positions.  What  firing  positions  and  other  body  positions  should  TTES 
be  able  to  simulate  for  hostiles.  neutrals,  and  friendlies,  as  trainees  decide  whether  to  fire? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  Total  Group  ratings  for  the  Potential  Usefulness  of  various  Adversary  Firing  Positions  are 
only  slightly  lower  than  Potential  Usefulness  ratings  for  the  same  factors  for  the  Trainee. 


Potential  Usefulness  Rating  for  Adversary 

Firing  Positions 

Total  Group 

Military 

Civilian 

Kneeling 

99.3 

99.1 

100.0 

Standing 

99.3 

99.1 

Prone 

98.5 

98.3 

100.0 

Squatting  or  crouched 

95.4 

94.5 

100.0 

Through  windows 

95.4 

94.5 

Through  doors 

95.4 

94.5 

100.0 

Supported 

94.4 

93.6 

100.0 

Sitting 

93.6 

92.7 

100.0 

From  holes 

93.6 

92.7 

100.0 

From  behind  trees 

93.6 

92.7 

100,0 

Barricaded 

91.7 

90.5 

100.0 

From  under  vehicles 

91.2 

90.0 

100.0 

On  back 

77.6 

74.5 

100.0 

Potential  Overall  TTES 
usefulness:  Firing  Positions 

93.6 

92.6 

100.0 

•  The  over-90%  ratings  for  12  of  the  13  Firing  Positions  places  all  of  these  simulation  factors 
except  On  back  (rated  substantially  lower)  in  the  critical  classification. 

•  Civilians  were  unanimous  in  giving  all  factors  a  top  rating,  resulting  in  values  of  100%  for  all 
Firing  Positions. 

•  In  1994,  this  category  received  a  Total  Group  Importance  rating  of  83.8%.  This  compares 
with  a  1995  93 .6%  Potential  Usefulness  rating,  about  a  12%  difference.  The  1995  experts 
obviously  want  to  practice  their  skills  against  adversaries  as  capable  as  will  be  found  in  real- 
world  combat. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  The  Total  Group  rated  six  of  the  listed  Firing  Positions  above  50%  in  Current  Capability. 
1994  and  1995  ratings  indicate  that  all  but  one  of  these  six  (Squatting  or  crouched)  have  been 
extensively  improved  in  the  past  year  (up  to  nearly  20%  better). 

•  Little  or  no  improvement  was  observed  for  the  remaining  seven  positions  (most  of  which  have 
not  been  modeled  in  TTES  at  all). 
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Firing  Positions 


Total  Group 


Current  Capability  Rating  for  Adversary 


1994  Total 


Military 


Civilian 


Standing 


Through  windows 


Through  doors 


Kneeling 


Prone 


Squatting  or  crouched 


Supported 


Sitting 


From  holes 


Barricaded 


From  behind  trees 


From  under  vehicles 


On  back 


Group 


64.7 


57.6 


58.8 


54.1 


51.8 


55.3 


36.3 


51.8 


44.7 


33.8 


33.8 


46.3 


40. 


Current  Overall  TTES  capability: 
Firing  Positions  _ 


•  The  Overall  Current  Capability  rating  of  56.5%  for  Adversary  Firing  Positions  compares  with 
46.2%  in  1994  —  about  a  22%  change. 

4.3.4  Arm  and  Hand  Movements 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 


Potential  Usefulness  Rating  for  Trainee 


Arm  &  Hand  Movements 

Total  Group 

Military 

Civilian 

Left  &  right 

97.6 

98.1 

95.0 

Diagonal  (up  &  to  the  right,  etc.) 

97.6 

98.1 

95.0 

Bend  elbows  &  wrists 

97.6 

98.1 

95.0 

Up  &  down 

96.9 

97.3 

95.0 

Open  doors 

96.9 

97.1 

96.0 

Open  windows 

96.9 

97.1 

96.0 

Manipulate  weapons 

96.8 

97.1 

95.0 

Draw  weapons 

96.7 

97.0 

95.0 

Transition  between  weapons 

96.7 

97.0 

95.0 

Shoot  weapons 

96.3 

96.4 

96.0 

Change  hands 

96.0 

96.2 

95.0 

Reach  for  objects 

95.8 

98.0 

85.0 

Grab  or  grasp  objects 

95.8 

98.0 

85.0 

Push  objects  away 

95.4 

97.1 

88.0 

Pick  up  objects 

95.0 

97.0 

85.0 

Throw  objects  (e.g.,  grenades) 

94.4 

95.2 

90.0 

Use  rifle  as  pugil 

90.4 

91.4 

85.0 

Potential  Overall  TTES  usefulness: 
Arm  &  Hand  Movements 

95.5 

96.5 

92.0 
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The  1995  Total  Group  rated  all  Arm  &  Hand  Movement  factors  above  90%  in  Potential 
Usefulness  for  the  Trainee.  All  must  be  considered  critical  capabilities  to  include  in  TTES. 


•  In  1994  this  category  received  a  Total  Group  overall  average  Importance  rating  of  86.4%  for 
the  Trainee.  The  1995  Potential  Usefulness  rating  of  95.5%  suggests  about  a  10%  increase  in 
perceived  importance  or  usefulness  as  TTES  realism  has  grown. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Only  the  ability  to  Shoot  Weapons  is  rated  above  50%  in  Current  Capability,  as  the  Trainee 
attempts  to  use  a  variety  of  Arm  &  Hand  Movements.  Respondents  obviously  feel  very 
constrained  by  the  present  system  that  tethers  them  to  sensing  devices  rather  than  allowing  free 
movement  as  in  the  real  world  —  and  in  the  envisioned  TTES  system. 


Current  Capability  Rating  for  Trainee 

Arm  &  Hand  Movements 

Total  Group 

Military 

Civilian 

1994  Total 
Group 

Shoot  weapons 

66.4 

69.5 

46.7 

47.8 

Manipulate  weapons 

45.0 

47.1 

30.0 

40.0 

Left  &  right 

41.4 

40.0 

50.0 

37.6 

Up  &  down 

41.3 

40.0 

50.0 

37.6 

Diagonal  (up  &  to  the  right,  etc.) 

33.8 

34.5 

30.0 

35.3 

Bend  elbows  &  wrists 

32.3 

32.7 

30.0 

32.9 

Draw  weapons 

31.4 

31.7 

30.0 

35.3 

Change  hands 

30.8 

30.9 

30.0 

32.9 

Use  rifle  as  pugil 

29.2 

29.1 

30.0 

33.8 

Throw  objects  (e.g.,  grenades) 

28.3 

28.0 

30.0 

27.1 

Open  doors 

27.7 

27.3 

30.0 

29.4 

Open  windows 

27.7 

27.3 

30.0 

32.9 

Transition  between  weapons 

27.7 

27.3 

30.0 

34.1 

Push  objects  away 

26.7 

26.0 

30.0 

28.2 

Reach  for  objects 

26.2 

25.5 

30.0 

30.6 

Grab  or  grasp  objects 

26.2 

25.5 

30.0 

28.2 

Pick  up  objects 

26.2 

25.5 

30.0 

29.4 

Current  Overall  TTES  capability: 
Arm  &  Hand  Movements 

49.0 

51.3 

40.0 

33.3 

•  Except  for  the  ability  to  Shoot  Weapons,  little  improvement  can  be  inferred  for  any  of  the  Arm 
&  Hand  Movement  factors  since  the  1994  survey  —  although  the  Overall  category  rating  does 
indicate  some  progress. 


4.b.  Simulated  adversary  arm  and  hand  movements.  What  tactically  correct  arm  and  hand 
movements  should  TTES  be  able  to  simulate  for  hostiles,  neutrals,  and  friendlies,  as  trainees 
go  through  training  situations? 
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Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  The  Total  Group  rated  all  Arm  &  Hand  Movements  better  than  90%  in  Potential  Usefulness 
for  simulated  Adversaries,  indicating  that  all  are  critical  to  include. 


Potential  Usefulness  Rating  for  Adversary 


Arm  &  Hand  Movements 

Total  Group 

Military 

Civilian 

Shoot  weapons 

95.6 

97.3 

88.0 

Manipulate  weapons 

95.4 

97.3 

85.0 

Up  &  down 

94.6 

96.4 

85.0 

Left  &  right 

94.6 

96.4 

85.0 

Bend  elbows  &  wrists 

94.6 

96.4 

85.0 

Open  doors 

94.6 

96.4 

85.0 

Draw  weapons 

94.6 

96.4 

85.0 

Reach  for  objects 

94.4 

96.2 

85.0 

Grab  or  grasp  objects 

94.4 

96.2 

85.0 

Pick  up  objects 

94.4 

96.2 

85.0 

Throw  objects  (e.g.,  grenades) 

93.8 

95.5 

85.0 

Change  hands 

93.1 

94.5 

85.0 

Open  windows 

93.1 

94.5 

85.0 

Use  rifle  as  pugil 

93.1 

94.5 

85.0 

Transition  between  weapons 

92.8 

94.3 

85.0 

Diagonal  (up  &  to  the  right,  etc.) 

92.3 

93.6 

85.0 

Push  objects  away 

92.3 

93.6 

85.0 

Potential  Overall  TTES  usefulness: 
Arm  &  Hand  Movements 

93.9 

95.6 

88.0 

•  Military  respondents  generally  gave  higher  ratings  to  these  items  than  did  the  Civilians. 

•  In  1994,  the  Total  Group  of  respondents  gave  an  average  Importance  rating  of  82.4%  to  this 
overall  category  for  Adversaries,  about  14%  lower  than  1995 ’s  93.9%  Overall  Potential 
Usefulness  rating. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Simulated  Adversary  Current  Capability  to  utilize  a  variety  of  Arm  &  Hand  Movements  is 
considered  poor  (with  the  possible  exception  of  Shoot  Weapons)  —  but  maybe  a  bit  better  than 
the  Trainee’s  movements.  The  Total  Group’s  Overall  rating  for  this  category  is  50.6%  for 
Adversary  Arm  &  Hand  Movements,  compared  with  49.0%  for  the  Trainee. 

•  The  group  of  Military  respondents  generally  rated  Current  Capability  of  the  Adversary  factors 
even  lower  than  did  the  Civilian  group. 

•  In  spite  of  continued  low  scores,  an  Overall  improvement  of  more  than  50%  can  be  inferred 
from  the  two  Adversary  Current  Capability  Total  Group  ratings,  33.3%  in  1994  and  50.6%  in 
1995. 


Current  Capability  Rating  for  Adversary 

Arm  &  Hand  Movements 

Total 

Group 

Military 

Civilian 

1994  Total 
Group 

Shoot  weapons 


Manipulate  weapons 


Up  &  down 


Left  &  right 
Draw  weapons 


Diagonal  (up  &  to  the  right,  etc.) 

31.7 

Bend  elbows  &  wrists 

30.0 

Reach  for  objects 

30.0 

Grab  or  grasp  objects 


Use  rifle  as  pugil 


Throw  objects  (e.g.,  grenades) 


Open  doors 


Open  windows 


Transition  between  weapons 


Pick  up  objects 


Change  hands 


Push  objects  away 


Current  Overall  TTES  capability: 
Arm  &  Hand  Movements 


5. a.  Trainee  head  movements.  What  tactically  correct  trainee  head  movements  are  trainees  likely 
to  perform  and  for  which  TTES  should  provide  a  simulated  field  of  regard? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 


Potential  Usefulness  Rating  for  Trainee 


Head  Movements 


Fast  tracking  of  moving  object 


Look  over  shoulder/behind 


Peek/look  over  obstacle 


Slow  scan  of  area 


Peek/look  around  corner 


Up  &  down 


Aim  through  sight 


Duck 


Left  &  right 


Bend  &  look 


Diagonal  (up  &  to  the  right,  etc.) 


Put  face  in  dirt 


Potential  Overall  TTES  usefulness: 
Head  Movements 


Total  Group 


Military 
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Except  for  one  item,  the  listed  Head  Movements  were  among  the  highest  rated  simulation 
factors  in  Potential  Usefulness  for  Trainees,  and  definitely  should  be  considered  critical  for 
TTES.  Trainees  do  not  want  their  heads  constrained. 


•  Put  face  in  dirt  is  considered  by  the  Total  Group  the  least  Potentially  Useful  Head  Movement 
for  Trainees  in  1995  (88.8%);  this  factor  also  came  in  last  in  this  category  on  the  1994 
survey. 

•  An  Importance  rating  of  89.8%  was  calculated  for  this  category  of  simulation  factors  in  1994, 
the  highest  of  any  for  Trainees.  The  1995  Potential  Usefulness  value  of  97.5%  also  is  the 
highest  rating  for  Trainee  factors  (and  second  only  to  Sounds  in  the  entire  survey).  The  ratings 
indicate  a  perceived  increase  of  about  8%  in  Head  Movement  importance  or  usefulness  over 
the  already  high  1994  rating. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Current  Capability  to  utilize  a  variety  of  Head  Movements  is  perceived  as  pretty  good  by  the 
Total  Group,  with  all  but  three  of  the  listed  factors  rated  above  50%. 


Current  Capability  Rating  for  Trainee 

Head  Movements 

Total  Group 

Military 

Civilian 

1994  Total 
Group 

Slow  scan  of  area 

60.8 

63.0 

50.0 

66.7 

Aim  through  sight 

59.2 

61.0 

50.0 

53.8 

Left  &  right 

57.6 

60.0 

45.0 

50.0 

Up  &  down 

55.5 

55.6 

55.0 

43.8 

Fast  tracking  of  moving  object 

54.5 

56.7 

45.0 

37.5 

Bend  &  look 

52.7 

54.4 

45.0 

50.0 

Duck 

51.6 

54.7 

40.0 

55.0 

Peek/look  over  obstacle 

50.4 

52.6 

40.0 

47.5 

Peek/look  around  corner 

50.0 

51.0 

45.0 

50.0 

Diagonal  (up  &  to  the  right,  etc.) 

47.4 

48.0 

45.0 

47.5 

Look  over  shoulder/behind 

37.1 

37.6 

35.0 

33.3 

Put  face  in  dirt 

27.1 

29.1 

20.0 

41.3 

Current  Overall  TTES  capability: 
Head  Movements 

52.7 

55.6 

40.0 

mm 

•  The  group  of  five  Civilians  was  less  impressed  with  TTES  Current  Capability  than  were  the 
Military  respondents,  for  this  simulation  category. 

•  Very  little  progress  is  perceived  from  1994,  with  the  Total  Group’s  Overall  Current 
Capability  rating  rising  only  from  50.0  to  52.7%. 


5.b.  Simulated  adversary  head  movements.  What  tactically  correct  head  movements  should 
TTES  be  able  to  simulate  for  hostiles,  neutrals,  and  friendlies? 
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Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  For  their  TTES  Adversaries,  the  Total  Group  of  respondents  feels  that  all  of  the  listed  factors 
(including  Put  face  in  dirt)  have  very  high  Potential  Usefidness ;  all  fall  in  the  critical- to- 
include  category. 


Potential  Usefulness  Rating  for  Adversary 

Head  Movements 

Total  Group 

Military 

Civilian 

Aim  through  sight 


Look  over  shouider/behind 


Peek/!ook  around  corner 


Peek/look  over  obstacle 


Duck 


Up  &  down 


Left  &  right 


Fast  tracking  of  moving  object 


Bend  &  look 


Diagonal  (up  &  to  the  right,  etc.) 


Slow  scan  of  area 


Put  face  in  dirt 


Potential  Overall  TTES 
usefulness:  Head  Movements 


•  The  Potential  Usefulness  of  the  listed  Head  Movements  is  perceived  as  slightly  more 
important  for  Trainees  (97.5%)  than  for  Adversaries  (93.6%). 

•  When  the  1994  Overall  Total  Group  Importance  rating  (81.8%)  is  compared  with  the  1995 
Potential  Usefulness  rating  (93.6%),  about  a  14%  increase  is  observed  from  last  year. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 


Current  Capability  Rating  for  Adversary 


1994  Total 
Group 


50.0 


Head  Movements 

Total  Group 

Military 

Civilian 

Left  &  right 


Slow  scan  of  area 


Fast  tracking  of  moving  object 


Duck 


Peek/look  around  corner 


Aim  through  sight 


Peek/look  over  obstacle 


Up  &  down 


Diagonal  (up  &  to  the  right,  etc. 


Bend  &  look 


Look  over  shoulder/behind 


Put  face  in  dirt 


Current  Overall  TTES  capability: 
Head  Movements 


•  Modeling  of  Adversary  Head  Movements  is  seen  as  very  poor  at  present,  with  all  listed 
factors  rated  below  50%  in  Current  Capability. 

•  The  Civilian  respondents  especially  down-rated  this  category. 

•  Respondents  judged  that  the  T rainee’s  Current  Capability  to  use  a  variety  of  Head 
Movements  in  TTES  scenarios  is  somewhat  better  than  the  modeled  Adversary’s  ability: 
52.7%  compared  with  48.0%. 

•  Although  the  difference  is  not  statistically  significant,  the  Total  Group’s  Current  Capability 
rating  for  Adversary  Head  Movements  has  dropped  2  points  since  1994.  This  may  indicate 
that  other  simulated  capabilities  have  improved  enough  so  that  the  simulated  Head 
Movements  now  are  perceived  as  less  well  modeled. 

4.3.6  Eye  Movements 

6.a.  Trainee  eye  movements.  What  tactically  correct  eye  movements  are  trainees  likely  to 
perform  and  for  which  TTES  should  provide  a  simulated  field  of  regard? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  The  Potential  Usefulness  of  Trainee  Eye  Movements  is  rated  surprisingly  high  (96.4%)  by  the 
Total  Group,  almost  as  high  as  Head  Movements  (97.5%).  All  listed  movements  fall  into  the 
critical  classification.  Glancing  around  obviously  is  considered  a  natural  (and  important) 
thing  to  do  during  training  exercises. 


Potential  Usefulness  Rating  for  Trainee 

Eye  Movements 

Total  Group 

Military 

Civilian 

Fast  tracking  of  moving  objects 

98.4 

98.0 

100.0 

Sight  alignment 

98.4 

98.0 

100.0 

Scan  area,  limited  head  motion 

97.7 

97.1 

100.0 

Through  peep  holes 

96.8 

96.0 

100.0 

Diagonal  (up  &  to  the  right,  etc.) 

95.4 

96.2 

92.0 

Up  &  down 

94.6 

93.3 

100.0 

Left  &  right 

94.1 

92.7 

100.0 

Potential  Overall  TTES  usefulness: 
Eye  Movements 

96.4 

95.3 

100.0 

•  In  1994,  the  Total  Group’s  average  Eye  Movement  Importance  rating  was  84.9%  for  the 
Trainee.  This  compares  with  the  1995  Potential  Usefulness  value  of  96.4%,  about  a  13% 
perceived  increase  in  importance  or  usefulness. 

Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  The  Trainee’s  Current  Capability  to  move  eyes  Up  &  down  and  Left  &  right  is  judged  fairly 
good  by  the  Total  Group,  with  Sight  alignment  and  Diagonal  movements  also  rated  above 
50%. 
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Current  Capability  Rating  for  Trainee 


_ 

Eye  Movements 

Total 

Group 

Military 

Civilian 

1994  Total 
Group 

Up  &  down 

61.9 

65.9 

45.0 

42.5 

Left  &  right 

61.0 

65.6 

33.3 

41.3 

Sight  alignment 

54.0 

58.8 

26.7 

40.0 

Diagonal  (up  &  to  the  right,  etc.) 

52.9 

57.1 

33.3 

42.5 

Fast  tracking  of  moving  objects 

46.3 

48.8 

33.3 

32.5 

Scan  area,  limited  head  motion 

46.0 

48.2 

33.3 

Through  peep  holes 

42.5 

46.2 

26.7 

32.5 

Current  Overall  TTES  capability: 

Eye  Movements 

51.6 

56.0 

35.0 

40.0 

•  Fast  tracking  and  Scan  area  probably  are  downrated  because  the  current  TTES  head-position 
tracker  has  difficulty  keeping  up  with  rapid  movements.  Objects  on  the  TTES  screen  cannot 
be  approached  close  enough  to  look  Through  peep  holes  at  present. 

•  The  Civilian  group  generally  rated  the  listed  Current  Capability  for  Eye  Movements  much 
lower  than  did  the  Military  Group.  Possibly  subtle  eye  movements  are  more  important  in  law 
enforcement  activities  than  in  out-and-out  combat. 

•  The  1995  Total  Group  perception  of  TTES  Overall  Current  Capability  for  Trainee  Eye 
Movements  (5 1.6%)  is  nearly  30%  higher  than  was  the  perception  by  the  1994  Total  Group 
(40.0%). 


6.b.  Simulated  adversary  eye  movements.  What  tactically  correct  eye  movements  should  TTES 
be  able  to  simulate  for  hostiles,  neutrals,  and  friendlies? 


Potential  Usefulness  of  Listed  Factors  for  TTES  Training 

•  The  Potential  Usefulness  of  the  Adversary  being  able  to  Scan  area  is  rated  at  90.0%  by  the 
Total  group,  just  into  the  critical  classification. 


Potential  Usefulness  Rating  for  Adversary 

Eye  Movements 

Total  Group 

Military 

Civilian 

Scan  area,  limited  head  motion 

90.0 

92.0 

70.0 

Fast  tracking  of  moving  objects 

88.6 

90.5 

70.0 

Through  peep  holes 

88.2 

90.0 

70.0 

Sight  alignment 

88.2 

90.0 

70.0 

Diagonal  (up  &  to  the  right,  etc.) 

87.8 

89.5 

70.0 

Up  &  down 

86.1 

87.6 

70.0 

Left  &  right 

86.1 

87.6 

70.0 

Potential  Overall  TTES  usefulness: 
Eye  Movements 

86.7 

88.8 

70.0 
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The  remaining  six  listed  Adversary  Eye  Movements,  rated  between  86%  and  89%,  should  be 
classed  as  extremely  important  to  include. 


•  The  Potential  Usefulness  of  Trainee  Eye  Movements  (96.4%)  is  rated  substantially  higher 
than  Adversary  Eye  Movements  (86.7%). 

•  The  1995  Total  Group  Potential  Usefulness  rating  of  86.7%  for  the  Adversary  Eye  Movement 
category  compares  with  a  calculated  average  Importance  rating  of  74.6%  in  1994,  indicating 
about  a  16%  increase  in  perceived  usefulness  or  importance. 

•  Implementing  the  great  range  of  Adversary  Eye  Movements  listed  here  (and  considered  useful 
by  the  subject  matter  experts)  will  be  extremely  difficult  to  do,  using  the  current  TTES  human 
entity  model. 


Current  Capability  of  TTES  to  Emulate  Listed  Factors 

•  Adversary  Left  &  right  Eye  Movements  are  the  only  listed  items  rated  above  50%  in  Current 
Capability  by  the  Total  Group  in  1995. 


Current  Capability  Rating  for  Adversary 

Eye  Movements 

Total  Group 

Military 

Civilian 

Left  &  right 

50.9 

50.9 

— 

41.3 

Sight  alignment 

49.1 

49.1 

— 

40.0 

Up  &  down 

48.0 

48.0 

— 

42.5 

Scan  area,  limited  head  motion 

43.3 

43.3 

— 

40.0 

Fast  tracking  of  moving  objects 

42.0 

42.0 

— 

32.5 

Diagonal  (up  &  to  the  right,  etc.) 

41.8 

41.8 

— 

42.5 

Through  peep  holes 

38.0 

38.0 

— 

32.5 

Current  Overall  TTES  capability: 

Eye  Movements 

54.5 

54.0 

mm 

mm 

•  None  of  the  five  Civilians  provided  ratings  for  the  individual  factors  in  this  particular  category. 

•  As  noted  above,  implementation  of  a  variety  of  Adversary  Eye  Movements  is  not  possible  in 
the  current  version  of  TTES,  and  will  be  very  difficult  (expensive)  to  do  in  the  future.  The 
relatively  high  ratings  for  items  in  this  category  (even  though  mostly  less  than  50%)  may 
indicate  that  the  survey  respondents  recognize  the  difficulty  of  implementing  the  listed 
Adversary  Eye  Movements  and  so  do  not  expect  very  great  capability. 

•  About  a  35%  increase  in  Current  Capability  over  the  past  year  can  be  inferred  from  the  1994 
and  1995  Total  Group  ratings.  This  is  surprising,  since  the  current  TTES  adversary  entities 
have  no  ability  to  move  their  eyes  apart  from  moving  the  entire  head.. 
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5.0  SUMMARY  RESULTS 


The  results  for  individual  questions  provided  above  now  can  be  summarized  and  reviewed  as  a 
whole.  The  following  section  includes  this  summarization,  discusses  the  results,  and  provides 
several  conclusions  that  should  be  considered  during  further  TTES  development. 

5.1  POTENTIAL  USEFULNESS  RATINGS 

A  total  of  28  subject  matter  experts  were  asked  to  evaluate  the  Potential  Usefulness  of 
numerous  individual  simulation  factors,  grouped  into  17  categories.  They  also  gave  an  overall 
rating  for  the  usefulness  of  each  category  as  a  whole.  The  overall  ratings  for  the  17  categories  are 
summarized  in  the  following  table  separately  for  the  Total  group  and  for  Military  and  Civilian 
respondents  (it  should  be  recalled  that  only  five  civilians  participated  in  the  1995  survey).  Average 
Importance  ratings  from  the  1994  survey  are  included  in  the  last  column,  for  comparison.  The 
Mission  Preview  and  Mission  Rehearsal  categories  were  not  included  in  the  1994  survey. 


Potential  Usefulness  or  Importance 

Simulation  Capability  Categories 

1995 

Total  Group 

1995 

Military 

1995 

Civilians 

1994  Total 
Group 

A.1  Training  Situations 

96.4 

95.6 

100.0 

76.4 

A.2.  Marksmanship  Events 

95.0 

95.0 

95.0 

85.0 

A.3.  Discretionary  Decisions 

95.2 

95.3 

95.0 

83.5 

A.4.  Mission  Preview 

94.8 

95.0 

93.3 

- 

A.5.  Mission  Rehearsal 

93.9 

94.7 

90.0 

- 

B.l.  Environments 

<0 

K> 

o 

90.6 

100.0 

81.9 

B.2.  Environmental  Effects 

92.6 

90.7 

100.0 

83.7 

B.3.  General  Objects 

96.8 

97.3 

95.0 

84.9 

B.4.  Simulated  Humans 

96.4 

95.6 

100.0 

92.7 

B.5.  Weapons 

90.6 

93.3 

81.8 

B.6.  Sounds 

98.0 

98.8 

95.0 

86.7 

C.l.  Tactical  Movements:  Trainee 

96.2 

95.3 

100.0 

85.2 

C.2.  Individual  Body  Movements;  Trainee 

95.2 

96.5 

90.0 

88.0 

C.3.  Firing  Positions:  Trainee 

96.0 

95.0 

100.0 

88.2 

C.4.  Arm  &  Hand  Movements:  Trainee 

95.5 

96.5 

92.0 

86.4 

C.5.  Head  Movements:  Trainee 

97.5 

96.8 

100.0 

89.8 

C. 6.  Eye  Movements:  Trainee 

96.4 

95.3 

100.0 

84.9 

C.l.  Tactical  Movements:  Adversary 

94.7 

97.3 

85.0 

74.6 

C.2.  Individual  Body  Movements:  Adversary 

94.3 

|  96.5 

85.0 

83.5 

C.3.  Firing  Positions:  Adversary 

93.6 

92.6 

100.0 

83.8 

C.4.  Arm  &  Hand  Movements:  Adversary 

93.9 

95.6 

88.0 

82.4 

C.5.  Head  Movements:  Adversary 

93.6 

93.3 

95.0 

81.8 

C.6.  Eye  Movements:  Adversary 

86.7 

88.8 

70.0 

74.6 

Averages 

94.6 

94.7 

94.0 

83.8 

5.1.1  TTES  Capabilities  for  Trainees  and  for  Simulated  Adversaries 


It  should  be  noted  that  Sections  A  and  B  of  the  questionnaire  asked  individual  questions 
related  to  simulator  design  (that  is,  the  kinds  of  situations,  environments,  and  objects  that  should  be 
emulated  in  TTES)  while  the  questions  in  Section  C  were  different.  Two  questions  were  asked  for 


each  category  in  Section  C:  (1)  what  capabilities  should  TTES  provide  for  Trainees,  and  (2)  what 
capabilities  should  TTES  simulate  for  the  Adversaries  with  which  the  trainees  will  interact. 

•  Overall,  the  respondents  reported  that  the  factors  in  the  six  categories  of  questionnaire  Section 
C  are  somewhat  more  useful  for  Trainees  than  for  simulated  Adversaries  (an  average  Potential 
Usefulness  value  of  96. 1%,  compared  with  92.6%).  However,  the  difference  is  not  as 
pronounced  as  in  1994  when  Trainee  Importance  ratings  averaged  87. 1%  while  the  average 
Adversary  rating  was  80. 1% 

•  All  of  1995  respondents  are  experienced  trainers.  This  provides  an  interesting  comparison 
with  the  1994  results.  At  that  time,  the  Total  Group  provided  an  average  Importance  rating  of 
87. 1%  for  all  of  the  Trainee  factors  while  the  subgroup  of  13  trainers  gave  an  average  rating 
of  97.8%  for  these  factors  —  quite  close  to  the  1995  rating  of  96. 1%. 

•  From  the  subject  matter  expert  judgments  it  is  obvious  that  the  final  TTES  system  should  not 
constrain  the  activities  of  its  users  any  more  than  is  absolutely  necessary.  Equipment  used  by 
trainees  should  allow  them  to  carry  out,  in  a  normal  manner,  those  procedures  and  activities 
that  are  common  during  combat  —  insofar  as  is  reasonable  to  emulate. 

•  Results  of  the  1994  survey  indicated  a  general  attitude  that  the  adversary  is  inherently  inferior 
(or  at  least  less  capable)  and  requires  a  less  sophisticated  representation  in  TTES.  It  is 
reassuring  that  no  such  attitude  is  observed  in  the  1995  survey  results.  Such  a  perception 
would  be  extremely  dangerous  in  real-world  combat;  nothing  encountered  during  training 
should  reinforce  this  misconception  in  any  way.  The  adversaries  engaged  during  TTES 
training  must  appear  to  be  bright,  clever,  and  fully  capable  of  carrying  out  well-planned  and 
skilled  coordinated  tactics. 

5.1.2  Military  Versus  Civilian  Ratings 

The  group  of  five  civilians  who  participated  in  the  study  is  so  small  that  little  validity  can  be 
given  to  their  average  results,  when  taken  alone.  For  some  individual  simulation  categories  the 
Civilian  group  responses  are  considerably  greater  or  smaller  than  those  for  the  Military 
respondents  and  for  the  Total  Group  (which  tracked  closely  due  to  the  preponderance  of  Military 
participants).  Yet,  as  may  be  observed  on  the  bottom  line  of  the  above  table,  the  overall  average 
results  for  the  Civilian  group  do  not  differ  in  any  significant  way  from  those  of  the  Total  Group. 

•  Although  the  Civilian  responses  cannot  be  considered  valid  representations  of  the  population  of 
civilian  law  enforcement  officers  (and  their  average  results  should  not  be  used  alone),  the 
group’s  survey  responses  are  generally  similar  to  those  of  the  Military  respondents  and  to  the 
Total  Group.  Inclusion  in  the  overall  survey  results  will  result  in  a  larger  sample  and  should 
not  bias  the  overall  results  substantially. 

5.1.3  Overall  Category  Usefulness 

The  relative  overall  Potential  Usefulness  of  the  categories  easily  can  be  examined  using  the 
information  from  the  above  table  if  it  is  sorted  so  that  the  highest-rated  categories  are  at  the  top. 
This  has  been  done  in  the  table  below. 
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Potential  Usefulness  or  Importance 

Simulation  Capability  Categories 

1995 

Total  Group 

1995 

Military 

1995 

Civilians 

1994  Total 
Group 

B.6.  Sounds 

98.0 

98.8 

95.0 

C.5.  Head  Movements:  Trainee 

97.5 

96.8 

100.0 

89.8 

B.3.  General  Objects 

96.8 

97.3 

95.0 

84.9 

B.4.  Simulated  Humans 

96.4 

95.6 

100.0 

92.7 

C.6.  Eye  Movements:  Trainee 

96.4 

95.3 

*  100.0 

84.9 

C.l.  Tactical  Movements:  Trainee 

96.2 

95.3 

100.0 

85.2 

C.3.  Firing  Positions:  Trainee 

96.0 

95.0 

100.0 

88.2 

C. 4.  Arm  &  Hand  Movements:  Trainee 

96.5 

92.0 

86.4 

A.1  Training  Situations 

95.4 

95.6 

76.4 

C.2.  Individual  Body  Movements:  Trainee 

95.2 

96.5 

■a 

mmm 

A.3.  Discretionary  Decisions 

95.2 

95.3 

95.0 

83.5 

A.2.  Marksmanship  Events 

95.0 

95.0 

85.0 

A.4.  Mission  Preview 

94.8 

93.3 

— 

C.l.  Tactical  Movements:  Adversary 

94.7 

97.3 

85.0 

74.6 

C.2.  Individual  Body  Movements:  Adversary 

94.3 

96.5 

85.0 

83.5 

C.4.  Arm  &  Hand  Movements:  Adversary 

93.9 

95.6 

88.0 

82.4 

A.5.  Mission  Rehearsal 

93.9 

94.7 

90.0 

— 

C.3.  Firing  Positions:  Adversary 

93.6 

92.6 

100.0 

83.8 

C.5.  Head  Movements:  Adversary 

93.6 

93.3 

95.0 

81.8 

B.2.  Environmental  Effects 

92.6 

90.7 

100.0 

83.7 

B.t.  Environments 

92.0 

90.6 

100.0 

81.9 

B.5.  Weapons 

91.0 

90.6 

93.3 

81.8 

C.6.  Eye  Movements:  Adversary 

86.7 

88.8 

70.0 

74.6 

Averages 

94.6 

94.7 

94.0 

83.8 

•  While  the  trend  for  the  top-rated  categories  (scoring  over  90%)  in  Potential  Usefulness  is 
interesting  (moving  from  Sounds  at  98.0%  down  to  Weapons  at  91.0%),  the  differences  are  too 
small  to  be  significant.  The  closeness  of  the  assigned  ratings  indicates  that  the  survey 
respondents  consider  inclusion  of  all  of  these  categories  critical. 

•  Eye  Movements  for  the  Adversary  is  rated  substantially  below  the  other  categories.  It  appears 
that  high-fidelity  simulation  of  the  facial  features  of  hostiles,  neutrals,  and  friendlies  is 
considered  less  important  than  is  modeling  of  numerous  battlefield  situations,  events,  and 
objects. 


•  The  ratings  indicate  that  in  1995  TTES  was  perceived  about  equally  useful  as  a  trainer  for 
Discretionary  Decisions  (95.2%)  and  Marksmanship  (95.0%).  This  is  a  major  change  from 
1994,  when  TTES  was  considered  primarily  a  Marksmanship  trainer  (85%)  and  to  a  lesser 
degree  a  Discretionary  Decisions  trainer  (83.5%).  The  changed  perception  no  doubt  is  due  to 
advances  in  TTES  implementation,  and  was  predicted  last  year.  In  1994,  all  simulated  humans 
were  adversaries  and  simply  were  targets  to  be  fired  at.  The  1995  version  of  TTES  is  more 
sophisticated,  includes  neutrals  and  friendlies  (fire  team  members)  as  well  as  hostiles,  and 
requires  trainees  to  think  before  they  shoot. 


5.2  CURRENT  CAPABILITY  RATINGS 

The  following  table  summarizes  the  survey  participants’  responses  regarding  how  capable 
they  consider  TTES  currently  is,  both  to  meet  Trainee  needs  for  various  scenarios  and  to  simulate 
objects  and  combat  environments.  Overall  ratings  for  the  1995  Total  Group  and  the  two  individual 
subgroups  are  provided  for  the  17  categories,  along  with  Total  Group  responses  from  1994.  Two 
simulation  categories.  Mission  Preview  and  Mission  Rehearsal,  were  not  included  on  the  1994 
questionnaires. 

In  1994  the  survey  respondents  were  not  asked  to  provide  separate  judgments  on  TTES 
capabilities  for  simulating  Adversaries  and  for  meeting  Trainee  needs;  a  single  judgment  served  to 
evaluate  both.  This  was  considered  a  weakness  of  the  1994  survey  and  was  corrected  on  the  1995 
questionnaire.  For  this  table,  the  1994  Total  Group  Current  Capability  ratings  simply  are 
repeated  for  Trainees  and  Adversaries. 


Current  Capability 

Simulation  Capability  Categories 

1995 

Total  Group 

1995 

Military 

1995 

Civilians 

1994  Total 
Group 

A.1  Training  Situations 

69.5 

71.8 

60.0 

45.0 

A.2.  Marksmanship  Events 

71.1 

71.4 

70.0 

46.3 

A.3.  Discretionary  Decisions 

70.0 

70.0 

70.0 

41.3 

A.4.  Mission  Preview 

72.6 

75.0 

60.0 

— 

A.5.  Mission  Rehearsal 

73,0 

72.9 

73.3 

- 

B.l.  Environments 

74.1 

74.7 

70.0 

54.3 

B.2.  Environmental  Effects 

54.1 

58.5 

40.0 

36.0 

B.3.  General  Objects 

64.4 

66.7 

53.3 

42.9 

B.4.  Simulated  Humans 

69.5 

67.8 

80.0 

42.9 

B.5.  Weapons 

63.2 

67.1 

30.0 

33.3 

B.6.  Sounds 

60.0 

65.3 

33.3 

30.7 

C.l.  Tactical  Movements:  Trainee 

58.0 

60.0 

50.0 

O 

o 

L. 

C.2.  Individual  Body  Movements:  Trainee 

60.0 

61.3 

53.3 

44.6 

C.3.  Firing  Positions:  Trainee 

57.3 

56.7 

60.0 

46.2 

C.4.  Arm  &  Hand  Movements:  Trainee 

49.0 

51.3 

40.0 

33.3 

C.5.  Head  Movements:  Trainee 

52.7 

55.6 

40.0 

50.0 

C.6.  Eye  Movements:  Trainee 

51.6 

56.0 

35.0 

O 

© 

C.l.  Tactical  Movements:  Adversary 

47.1 

49.1 

40.0 

40.0 

C.2.  Individual  Body  Movements:  Adversary 

58.9 

64.0 

33.3 

44.6 

C.3.  Firing  Positions:  Adversary 

56.5 

57.3 

50.0 

46.2 

C.4.  Arm  &  Hand  Movements:  Adversary 

50.6 

50.0 

53.3 

33.3 

C.5.  Head  Movements:  Adversary 

48.0 

55.0 

20.0 

50.0 

C.6.  Eye  Movements:  Adversary 

54.5 

54.0 

60.0 

40.0 

Averages 

60.2 

62.2 

51.1 

42.0 

•  The  most  obvious  result  is  not  unexpected:  evaluators  say  that  the  1995  iteration  of  TTES  is 
considerably  more  capable  than  the  1994  version  for  training. 

•  The  Total  Group  gave  a  reasonable  average  Overall  Current  Capability  rating  in  1995  for 
TTES,  60.2%.  This  is  up  over  18  points  (more  than  40%)  from  last  year. 
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The  Military  respondents  generally  gave  higher  Current  Capability  ratings  than  the  Civilians, 
no  doubt  due  to  the  current  TTES  emphasis  on  military'  urban  terrain  scenarios. 


Which  categories  of  capabilities  are  considered  right  now  to  be  the  best ?  The  same 
information  shown  in  the  table  above  is  provided  below,  but  sorted  so  that  the  categories  with  the 
largest  average  ratings  are  at  the  top. 


Current  Capability 

Simulation  Capability  Categories 

1995 

Total  Group 

1995 

Military 

1995 

Civilians 

1994  Total 
Group 

B.l.  Environments 

74.1 

74.7 

70.0 

54.3 

A.5.  Mission  Rehearsal 

72.9 

73.3 

— 

A.4.  Mission  Preview 

75.0 

60.0 

— 

A.2.  Marksmanship  Events 

71.1 

71.4 

70.0 

46.3 

A.3.  Discretionary  Decisions 

70.0 

70.0 

70.0 

41.3 

A.1  Training  Situations 

69.5 

71.8 

60.0 

45.0 

B.4.  Simulated  Humans 

69.5 

67.8 

80.0 

42.9 

B.3.  General  Objects 

64.4 

66.7 

53.3 

42.9 

B.5.  Weapons 

63.2 

33.3 

C.2.  Individual  Body  Movements:  Trainee 

60.0 

61.3 

53.3 

44.6 

B.6.  Sounds 

60.0 

33.3 

30.7 

C.2.  Individual  Body  Movements:  Adversary 

58.9 

64.0 

33.3 

44.6 

C.l.  Tactical  Movements:  Trainee 

58.0 

60.0 

C.3.  Firing  Positions:  Trainee 

57.3 

56.7 

60.0 

46.2 

C.3.  Firing  Positions:  Adversary 

56.5 

57.3 

50.0 

46.2 

C.6.  Eye  Movements:  Adversary 

54.5 

54.0 

60.0 

KEX91 

B.2.  Environmental  Effects 

54.1 

58.5 

40.0 

wgrrviBi 

C.5.  Head  Movements:  Trainee 

55.6 

40.0 

■SOI 

C.6.  Eye  Movements:  Trainee 

51.6 

56.0 

35.0 

C.4.  Arm  &  Hand  Movements:  Adversary 

50.6 

50.0 

53.3 

C. 4.  Arm  &  Hand  Movements:  Trainee 

49.0 

51.3 

40.0 

33.3 

C.5.  Head  Movements:  Adversary 

48.0 

55.0 

20.0 

50.0 

C.l.  Tactical  Movements:  Adversary 

47.1 

49.1 

40.0 

40.0 

Averages 

60.2 

62.2 

51.1 

42.0 

•  As  in  1994,  the  Environments  category  is  given  the  top  1995  Current  Capability  rating 

(74. 1%).  TTES  is  perceived  as  able  to  emulate  a  variety  of  environments  useful  for  weapons 
use  training. 

•  All  categories  except  Adversary  Head  Movements  are  judged  better  in  1995  than  in  1994  and 
all  except  Trainee  Arm  &  and  Movements  and  Adversary  Head  and  Tactical  Movements  are 
rated  above  50%  by  the  Total  Group. 

•  TTES  capability  to  simulate  Environmental  Effects  (weapon  impacts,  reduced  visibility,  etc.) 
may  be  lagging  most  other  categories,  as  it  also  was  perceived  to  be  in  1994. 

•  Judgments  of  Current  Capability  for  both  Trainee  and  Adversary  Firing  Positions  and  Body, 
Eye,  Arm  &  Hand,  Head,  and  Tactical  Movements  remain  somewhat  disappointing.  All  are 
rated  by  the  Total  Group  at  60%  or  less  in  1995. 
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•  Simulated  Adversary  Tactical  Movements  and  Head  Movements  are  rated  at  the  bottom  (47. 1 
and  48.0%)  of  the  Current  Capability  list.  This  simulation  category  will  be  difficult  and 
expensive  to  improve,  due  to  the  complexity  of  emulating  human  activities,  behavior,  and 
performance. 

5.3  HOW  CLOSE  IS  TTES  NOW? 

A  simple  ratio  was  used  to  determine  respondents’  perceptions  of  how  close  TTES  comes  at 
present  to  meeting  its  potential:  Total  Group  overall  Current  Capability  average  values  were 
divided  by  Total  Group  Potential  Usefulness  average  values  and  the  result  multiplied  by  100,  for 
each  of  the  17  categories.  Individual  category  ratios  are  listed  in  the  following  table,  sorted  so  the 
highest  category  ratio  is  at  the  top.  An  average  ratio  for  all  categories  combined  is  provided  at  the 
end  of  the  table. 

Similar  ratios  calculated  for  the  1994  results  also  are  included  in  the  final  column  for 
comparison.  The  Mission  Preview  and  Mission  Rehearsal  categories  were  not  included  in  the  1994 
survey.  Also,  separate  ratios  were  not  calculated  for  the  Section  C  Trainee  and  Adversary  results 
for  the  1994  survey  (a  weakness  of  that  study),  so  the  1994  Section  C  ratios  simply  are  duplicated 
in  the  final  column. 


Current-to-Potential  Ratios  j 

Simulation  Capability  Categories 

Current 

Capability 

Potential 

Usefulness 

1995 

Ratio 

1994  Ratio 

B.l.  Environments 

74.1 

92.0 

81% 

63% 

A.5.  Mission  Rehearsal 

73.0 

93.9 

78% 

- 

A. 4.  Mission  Preview 

72.6 

94.8 

77% 

- 

A.2.  Marksmanship  Events 

71.1 

95.0 

75% 

58% 

A.3.  Discretionary  Decisions 

70.0 

95.2 

74%  i 

48% 

A.1  Training  Situations 

69.5 

95.4 

73% 

59% 

B.4.  Simulated  Humans 

69.5 

96.4 

72% 

52% 

B.5.  Weapons 

63.2 

91.0 

69% 

41% 

B.3.  General  Objects 

64.4 

96.8 

67% 

49% 

C.6.  Eye  Movements:  Adversary 

54.5 

86.7 

63% 

54% 

C.2.  Individual  Body  Movements:  Trainee 

60.0 

95.2 

63% 

51% 

C.2.  Individual  Body  Movements:  Adversary 

58.9 

94.3 

62% 

51% 

B.6.  Sounds 

60.0 

98.0 

61% 

38% 

C.3.  Firing  Positions:  Trainee 

57.3 

96.0 

60% 

53% 

C.3.  Firing  Positions:  Adversary 

56.5 

93.6 

60% 

53% 

C.l.  Tactical  Movements:  Trainee 

58.0 

96.2 

60% 

46% 

B.2.  Environmental  Effects 

54.1 

92.6 

58% 

44% 

C. 5.  Head  Movements:  Trainee 

52.7 

97.5 

54% 

59% 

C.6.  Eye  Movements:  Trainee 

51.6 

96.4 

54% 

54% 

C.4.  Arm  &  Hand  Movements:  Adversary 

50.6 

93.9 

54% 

44% 

C.5.  Head  Movements:  Adversary 

48.0 

93.6 

51% 

59% 

C. 4.  Arm  &  Hand  Movements:  Trainee 

49.0 

95.5 

51% 

44% 

C.l.  Tactical  Movements:  Adversary 

47.1 

94.7 

50% 

46% 

Averages 

60.2  |  94.6 

64% 

51% 

•  From  the  ratios  it  can  be  inferred  that  the  subject  matter  experts  who  participated  in  the  1995 
study  consider  that  TTES,  as  a  whole,  is  about  64%  of  the  way  towards  meeting  its  potential, 
up  from  a  ratio  of  5 1%  in  1994. 
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•  Each  of  the  individual  simulation  categories  is  perceived  as  being  at  least  half  way  there  — 
with  a  ratio  of  50%  or  higher. 

•  The  Environments  category,  considered  to  have  the  greatest  Current  Capability,  is  considered 
closest  to  completion  (81%),  with  Mission  Rehearsal  (78%),  Mission  Preview  (77%),  and 
Marksmanship  Events  (75%)  all  about  three-quarters  of  the  way  towards  meeting  their 
potential.  Discretionary  Decisions,  Training  Situations,  and  Simulated  Humans  also  achieved 
high  current-to-potential  ratios  (above  70%). 

•  Emulation  of  Sounds,  highest  in  1995  Potential  Usefulness  ratings  and  in  the  middle  of  the 
Current  Capability  list,  has  come  a  long  way  this  year,  moving  from  the  bottom  place  ratio  of 
38%  in  1994  to  61%  in  1995  (a  60%  change).  Sound  capability  was  added  in  1995,  when 
audio-equipped  helmets  were  incorporated  into  TTES. 

5.4  HOW  GOOD  DOES  TTES  HAVE  TO  BE? 

New  in  1995  were  questions  on  the  Realism  Levels  that  are  required  for  satisfactory  training: 
how  good  is  “good  enough”?  These  questions  were  included  only  for  Section  B  of  the  survey  form, 
concerning  objects,  situations,  and  events.  The  instructions  indicated  that  a  goal  is  to  keep 
development  costs  down,  so  respondents  would  not  simply  opt  for  perfection  in  all  areas.  Results 
are  summarized  in  the  following  table,  sorted  with  the  highest  Realism  Level  ratings  at  the  top. 


Required  Realism  Level  Ratings  | 

Simulation  Capability  Categories 

Total  Group 

Military 

Civilians 

B.3.  General  Objects 

79.3 

78.2 

84.0 

B.5.  Weapons 

78.5 

76.4 

88.0 

B.6.  Sounds 

76.9 

76.2 

80.0 

B.4.  Simulated  Humans 

74.8 

73.6 

80.0 

B.l.  Environments 

73.8 

73.3 

76.0 

B.2.  Environmental  Effects 

68.5 

67.6 

72.0 

Averages 

75.3 

74.2 

80.0 

•  Respondents  gave  surprisingly  high  ratings  for  the  level  of  realism  they  feel  TTES  must 
provide.  The  overall  average  calculated  rating  of  75.3%  is  below  the  80%  cutoff  for  Exact 
replication ,  but  very  near  the  top  of  the  classification  referred  to  as  Good  replication,  hard  to 
tell  it  is  not  real  (see  Sections  3.3  and  3.6). 

•  This  result  could  have  serious  implications  for  the  cost  of  TTES  development,  if  potential 
users  actually  will  require  the  indicated  level  of  realism.  However,  we  feel  that,  though  real- 
world  realism  may  be  strongly  desired,  the  Current  Capability  ratings  indicate  that  simulations 
on  the  level  of  Reasonable  replication,  appearance  about  as  expected,  will  meet  a  lot  of 
training  needs  that  presently  cannot  be  met  at  all.  Nothing  in  TTES  is  replicated  exactly  at 
present,  and  many  objects  barely  would  be  considered  Good  replications',  mostly  they  are 
Reasonable  models  of  what  they  represent.  Yet  the  1995  survey  respondents  have  indicated 
that  TTES  Current  Capability  is  reasonably  good  as  the  system  is  implemented  now! 
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6.0  CONCLUSIONS  AND  RECOMMENDATIONS 

As  was  explained  in  Section  4.0,  the  questionnaire  administered  to  28  subject  matter  experts 
was  divided  into  three  sections.  Each  section  addressed  a  different  level  of  simulation  fidelity: 
first,  overall  system  usefulness  structure,  and  capabilities;  second,  modeling  of  objects  and  events; 
and,  third,  detailed  modeling  of  humans.  Conclusions  related  to  the  three  survey  sections  are 
provided  below,  along  with  a  few  recommendations.  All  conclusions  are  based  on  Total  Group 
responses. 

6.1  OVERALL  TTES  USEFULNESS 

Section  A  of  the  survey  form  included  five  categories  of  questions.  These  categories  were  (1) 
current  and  planned  Training  Situations,  (2)  training  events  related  to  Marksmanship,  (3)  events 
related  to  making  Discretionary  Decisions  about  appropriate  responses  to  possible  adversaries, 

(4)  capabilities  needed  so  TTES  can  be  used  for  Mission  Preview,  and  (5)  activities  to  include  to 
use  TTES  for  Mission  Rehearsal  (these  last  two  categories  were  added  for  1995).  These  five 
categories  concern  overall  TTES  system  structure  and  operation,  the  basics  upon  which  the 
categories  included  in  the  follow-on  sections  of  the  survey  must  be  built.  These  categories  are 
integral  to  TTES  development. 

•  The  Total  Group  rated  Training  Situations  highest  of  the  five  categories  in  Potential 
Usefulness,  yet  it  rated  TTES’s  Current  Capability  in  this  area  the  lowest  of  the  five  (though 
still  markedly  improved  over  the  1994  version).  The  Current-Capability-to-Potential- 
Usefulness  ratio  also  is  the  lowest  of  this  group  of  categories. 

•  Marksmanship  and  Discretionary  Decisions  have  switched  places  in  perceived  importance  or 
usefulness  between  1994  and  1995.  While  ratings  of  both  have  increased  substantially,  the 
latter  now  is  rated  slightly  higher  than  the  former.  This  may  be  due  to  the  greater  fidelity  of 
human  representations  in  TTES.  It  now  is  easier  to  envision  using  the  system  to  decide  when 
to  engage  (instead  of  simply  shooting  whatever  comes  into  view). 

•  The  perceived  usefulness  of  TTES  in  training  for  Discretionary  Decisions  cannot  be 
overemphasized.  The  responses  of  the  surveyed  subject  matter  experts  gave  numerous 
indications  of  how  unique  and  useful  this  function  will  be. 

•  Using  TTES  for  Mission  Preview  and  Mission  Rehearsal,  not  considered  separately  in  1994, 
was  rated  as  a  very  useful  function  by  the  1995  respondents.  These  new  uses  for  TTES  show 
much  promise. 

•  The  perceived  Current  Capability  of  this  entire  set  of  five  categories  has  improved  materially 
in  the  past  year,  more  than  either  of  the  other  two  sets.  The  progress  made  by  the  TTES 
developers  has  not  been  lost  on  the  subject  matter  experts  who  participated  in  this  study. 

•  The  average  Current-to-Potential  ratio  for  these  five  categories  is  over  75%.  This  might  be 
interpreted  to  mean  that  TTES  is  three-quarters  of  the  way  complete  in  representing  the  factors 
included  in  Section  A  of  the  questionnaire. 
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•  From  the  above  points,  one  possibly  could  conclude  that  the  gross  development  of  TTES 
simulations  already  is  quite  satisfactory;  this  conclusion  would  be  in  error.  Even  though  the 
respondents  are  impressed  with  the  current  TTES  level  of  development,  they  also  have  made  it 
clear  that  many  individual  capabilities  they  consider  potentially  useful  for  training  still  have  a 
long  way  to  go! 

6.2  TTES  OBJECTS,  SITUATIONS,  AND  EVENTS 

Section  B  of  the  1995  survey  form  included  six  categories  of  questions  related  to  objects, 
situations,  and  events.  These  were  (1)  Environments,  (2)  Environmental  Effects,  (3)  General 
Objects,  (4)  Simulated  Humans,  (5)  Weapons,  and  (6)  Sounds.  These  categories  of  factors 
define  in  detail  how  to  model  the  five  general  simulation  categories  included  in  Section  A  of  the 
survey  form. 

•  As  in  1994,  TTES  ability  to  emulate  a  variety  of  Environments  received  the  highest 
calculated  1995  Current  Capability  rating  of  all  survey  categories,  and  its  Current-to- 
Potential  ratio  also  was  the  highest. 

•  Current  Capability  ratings  were  low  for  Environmental  Effects.  With  calculated  Potential 
Usefulness  ratings  placing  it  in  the  critical  classification,  this  category  has  a  relatively  low 
Current-to-Potential  ratio  and  deserves  more  attention  when  funding  permits. 

•  The  importance  given  to  Sounds  in  both  1994  and  1995  is  very  interesting.  In  last  year’s 
report  we  suggested  that  the  total  absence  of  sounds  in  the  demonstration  system  magnified 
their  importance.  This  year,  even  though  verbal  commands  have  been  added  to  the  scenarios 
(and  the  perceived  Current  Capability  rating  increased  almost  100%),  the  importance  of  this 
category  grew  to  where  it  was  the  highest  rated  in  Potential  Usefulness. 

•  Simulated  Humans,  while  still  perceived  by  study  participants  as  critical  factors  to  model, 
now  must  compete  with  General  Objects  for  second  or  third  place  (after  Sounds)  in  Potential 
Usefulness.  Great  progress  has  been  made  this  past  year  in  modeling  humans  on  the 
battlefield;  both  the  Current  Capability  rating  and  the  Current-to-Potential  ratio  are  higher 
for  Simulated  Humans  than  for  General  Objects  in  1995  (though  considerable  improvement 
is  perceived  in  modeling  other  objects  as  well  as  humans).  The  current  TTES  emphasis  on 
improving  the  fidelity  of  its  Sounds,  General  Objects,  and  Simulated  Humans  is  well  placed, 
and  development  should  continue. 

•  Weapons  once  again  were  at  the  bottom  of  this  category  group  (although  not  by  much)  in 
Potential  Usefulness.  This  may  be  an  indication  that  TTES’s  unique  capability  to  train  in 
Discretionary  Decisions  is  recognized  by  trainers  as  its  most  important  contribution  to  combat 
readiness,  reducing  slightly  the  value  of  including  a  large  variety  of  weapons. 

•  The  average  Current-to-Potential  ratio  for  the  six  categories  of  objects,  situations,  and  events 
is  68%  —  possibly  interpretable  as  being  two-thirds  of  the  way  towards  what  is  needed  for  a 
fully  useful  system.  However,  the  required  Realism  Level  rating  for  this  group  of  categories 
suggests  that  potential  TTES  users  really  would  like  to  see  modeling  that  is  much  closer  to  the 
real  world  than  is  currently  evidenced  in  present  simulation  scenarios. 
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6.3  TTES  TRAINEE  MOVEMENTS  AND  POSITIONS 

Section  C  of  the  survey  asked  participants  to  rate  factors  in  six  categories  for  their  effects  on 
the  Trainees  who  wall  use  TTES:  (1)  Coordinated  Tactical  Movements,  (2)  Individual  Body 
Movements,  (3)  Firing  Positions,  (4)  Arm  &  Hand  Movements,  (5)  Head  Movements,  and  (6) 
Eye  Movements.  These  categories  of  capabilities  define  the  allowed  Trainee  postures  and  how 
Trainees  should  be  able  to  move,  within  the  bounds  of  the  TTES  structure  and  environments  listed 
in  the  categories  included  in  the  other  two  survey  sections. 

•  All  six  categories  in  this  section  were  rated  above  95%  in  Potential  Usefulness  for  Trainees 
and  were  included  among  the  top  10  ratings  for  all  23  survey  categories  (counting  the  Trainee 
and  Adversary  categories  separately). 

•  For  full  acceptance,  TTES  eventually  must  allow  Trainees  to  move  and  act  as  if  they  actually 
were  in  combat  situations.  To  the  greatest  extent  possible  their  Coordinated  Tactical 
Movements,  Individual  Body  Movements,  and  Firing  Positions  should  not  be  artificially 
constrained  by  equipment  limitations.  This  may  provide  a  major  challenge  for  TTES 
developers. 

•  The  Total  Group’s  overall  rating  for  the  Potential  Usefulness  of  Eye  Movements  was  a 
surprisingly  high  96.4%  for  Trainees  —  slightly  higher  than  the  ratings  for  any  other  category 
in  the  group  except  the  highest-rated  Head  Movements. 

•  Current  Capability  for  Trainees  may  be  lagging  in  this  group  of  categories,  especially  for 
Arm  &  Hand  Movements  (rated  at  only  49%). 

•  The  average  1995  Current-to-Potential  ratio  for  the  six  Trainee  categories  in  this  group  was 
57%.  In  1994  the  ratio  was  5 1%.  Progress  is  evident,  but  not  as  pronounced  as  for  the  other 
two  sets  of  categories. 

6.4  TTES  SIMULATED  ADVERSARY  MOVEMENTS  AND  POSITIONS 

Paralleling  the  questions  related  to  Trainee  movements  and  positions.  Section  C  of  the  survey 

also  asked  participants  to  rate  factors  in  six  categories  that  affect  the  representation  of  simulated 
Adversaries:  (1)  Coordinated  Tactical  Movements,  (2)  Individual  Body  Movements,  (3)  Firing 
Positions,  (4)  Arm  &  Hand  Movements,  (5)  Head  Movements,  and  (6)  Eye  Movements.  These 
categories  determine  the  detail  with  which  simulated  humans  should  be  modeled  for  TTES  training. 

•  Modeling  simulated  adversaries  in  reasonably  realistic  detail  is  considered  more  important  by 
1995  respondents  than  by  those  in  1994.  These  results  alleviate  the  1994  concerns  about  a 
general  attitude  that  the  adversary  is  inherently  less  capable  and  requires  a  less  sophisticated 
representation  in  TTES.  Such  a  perception  would  be  extremely  dangerous  in  real-world 
combat.  Nothing  in  training  must  be  allowed  to  reinforce  this  misconception  in  any  way.  The 
adversaries  engaged  during  TTES  training  must  appear  to  be  bright,  clever,  and  fully  capable 
of  carrying  out  well-planned  and  skilled  Coordinated  Tactical  Movements.  Neutrals  must  be 
realistic  enough  so  intelligent  judgments  are  needed  when  deciding  whether  to  engage. 
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•  For  simulated  Adversaries,  all  but  one  of  the  six  categories  in  this  section  (Eye  Movements) 
were  rated  above  90%  in  Potential  Usefulness.  Yet,  even  with  this  high  rating,  all  six  fell 
among  the  bottom  10  Usefulness  ratings  for  all  23  survey  categories  (counting  the  Trainee  and 
Adversary  categories  separately). 

•  As  in  1994,  these  six  categories  are  considered  less  useful  or  important  to  implement  for 
Adversaries  than  they  are  for  Trainees  —  though  by  a  much  smaller  margin  than  a  year  earlier. 

•  Current  Capability  ratings  for  these  Adversary  categories  generally  were  among  the  poorest 
for  the  entire  1995  survey.  When  the  Total  Group’s  Trainee  and  Adversary  ratings  for  the 
individual  categories  are  directly  compared,  TTES  capability  for  all  categories  was  rated  lower 
for  simulated  humans  than  for  Trainees,  except  for  Arm  &  Hand  Movements  and  Eye 
Movements.  The  gap  was  widest  for  Tactical  Movements. 

•  Average  factor  rating  for  the  Potential  Usefulness  of  Eye  Movements  was  96.4%  for 
Trainees  and  86.7%  for  simulated  Adversaries.  This  level  of  fidelity  for  computer-controlled 
humans  may  be  perceived  as  not  terribly  important  for  a  training  system  such  as  TTES. 

•  The  average  1995  Current-to-Potential  ratio  for  the  six  Adversary  categories  in  this  group 
was  57%,  the  same  as  for  Trainees.  In  1994  the  values  were  5 1%  for  both  the  Trainee  and  the 
Adversary.  Some  progress  is  perceived,  but  not  as  much  as  for  the  other  two  sets  of 
categories. 

6.5  FINAL  THOUGHTS  FOR  1995 

The  experts’  1994  overall  judgment  of  TTES  bears  repeating.  In  that  first  survey,  a  large 
majority  of  respondents  reported  that  they  expect  the  quality  of  TTES  training  to  be  in  the  Good- 
to-Very-Good  category  when  the  system  is  completed.  The  Total  Group  of  study  participants 
agreed  that  there  is  a  need  for  such  a  system,  and  they  generally  concurred  that  TTES  will  meet 
more  than  80%  of  that  need.  In  fact,  the  1994  subgroup  of  Trainers  gave  TTES  a  Potential 
Capability  value  of  85%  (as  reported  in  last  year’s  results). 

The  results  for  1995  (when  all  participants  were  experienced  small-arms  trainers)  are  even 
stronger:  the  Total  Group  rating  for  overall  TI  ES  Potential  Usefulness  was  calculated  to  be 
greater  than  94%.  If  Potential  Capability  (the  1994  term)  and  Potential  Usefulness  (used  in 
1995)  can  be  considered  very  similar  concepts,  the  latest  group  of  TTES  evaluators  feel  that  the 
system  will  come  even  closer  to  meeting  military  and  law  enforcement  training  needs  than  was 
perceived  by  the  1994  experts. 

If  TI  ES  can  provide  a  high  quality  trainer  that  meets  over  90%  of  military  and  civilian  needs 
for  a  discretionary  decision  and  marksmanship  training  system,  it  could  result  in  phenomenal 
savings  in  time,  ammunition,  and  other  expenses  —  and  should  pay  back  its  development  costs  in  a 
very  short  time!  Even  more  important,  a  system  such  as  TTES  will  provide  military  and  civilian 
agencies  with  the  capability  to  train  to  higher  standards  and  to  challenge  trainees  in  ways  not 
currently  possible. 
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Appendix  A:  1995  SURVEY  FORM 
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TTES:  Capability  and  Usefulness  for  Training 


Respondent  Information 


Name: 


Agency  and  Mailing  Address: 


Phone  Number: 


Trainer/Instructor?  Yes  No 


No.  of  Years: 


Instructions 

•  TTES  will  be  used  as  a  training  system  for  two  combat  functions: 

1.  Discretionary  decision  making:  deciding  whether  a  simulated  person  is  hostile,  neutral,  or 
friendly  and  whether  to  engage,  and  making  the  appropriate  response. 

2.  Marksmanship:  enhancing  precision  shooting  skills. 


•  Please  rate  each  item  in  the  following  lists: 

1 .  Indicate  how  capable  the  current  TTES  version  appears  to  be. 

2.  Indicate  how  useful  the  planned  2002  TTES  version  will  be. 

•  Use  the  following  linear  5-point  scale  to  rate  each  question: 


TTES  current 


1  Very  good 

2  Good 

3  Borderline 

4  Poor 

5  Very  poor 


mmm 


TTES  potential  usefulness 


1  Very  useful 

2  Somewhat  useful 

3  Borderline 

4  Somewhat  useless 

5  Very  useless 


Please  do  not  simply  indicate  that  every  item  is  Very  useful .  Not  all  capabilities  can  be  implemented  at 
once,  so  we  need  to  know  which  are  most  critical  to  include  first. 

•  For  Section  B,  indicate  how  realistic  TTES  simulations  must  be,  for  satisfactory  training 
while  keeping  costs  down.  How  good  is  “good  enough”? 


Thank  you  for  your  assistance! 

Your  written  and  verbal  comments  are  welcome! 


Return  survey  forms  to: 


MAJ  Frank  Wysocki 

Amphibious  Warfare  Technology  Directorate 
US  Marine  Corps  Systems  Command 
Quantico,  VA  22134 


(703)  640-2220  or  4788 


A.  Overall  TTES  Usefulness 


1.  TTES  training  situations.  What  training  situations  should  be  included  in  the  TTES  scenarios 


synthetic  environment,  for  maximum  TI  ES  usefulness  as  a  training  technology? 


TTES  Training  Situation: 

wmwmmm* 

Potential  Usefulness 

1.  Anti-armor/artillery  missions 

2.  Immediate  action  drills 

3.  MOUT  patrolling  operations 

4.  Combined  arms  operations 

5.  Neutral  evacuation  operations 

6.  Raid  operations 

7.  Ambush  operations 

8.  Recon  operations 

9.  Demolition/EOD  operations 

10.  Squad  &  fire  team  tactics 

11.  Obstacle  breaching  tactics 

12.  Fighting  hole  tactics 

13.  Advance-through-terrain 
activities 

14.  Advance-through-urban  areas 

15.  Sniping  activities 

16.  Building  clearing 

17.  Room  clearing 

18.  Close-quarter-battle 

19.  Perimeter  defense 

20.  Trench  warfare 

2 1 .  Hostage  rescue  team  activities 

22.  Guard  duty 

23.  Care  for  wounded 

24.  Unit-to-unit  coordination 

23.  Small  unit  leadership  practice 

26.  Land  navigation  practice 

27.  Combat  rehearsals 

28.  Other  (specify) 

29.  Overall  TTES  capability: 
Training  Situations 

2.  TTES  events:  marksmanship.  What  events  should  be  included  in  the  TTES  scenarios 


synthetic  environment  to  aid  in  marksmanship  training? 


Marksmanship  Events 

1.  Close-range  shots 

2.  Long-range  shots 

3.  Targets  at  unknown  ranges 

4.  Quick  aim/fire 

5.  Precision  shots  at  small  targets 

6.  Sniping 

7.  Multiple  targets,  multiple  shots 

8.  Sustained  rate  of  fire 

9.  Track  moving  targets 

10.  Fire  at  moving  targets 

11.  Awkward  shooting  positions 

12.  Indoor  shots 

13.  Shoot  while  moving 

14.  Reload  weapon 

|  15.  Clear  weapon  malfunctions 

16.  Change  weapons  (rifle/pistol) 

17.  Throw  grenades 

18.  Other  (specify) 

19.  Overall  TTES  capability: 

Marksmanship  Events _ 


TTES  events:  discretionary  decisions.  What  events  should  be  included  in  the  TTES  scenarios 
and  synthetic  environment  to  aid  in  making  decisions  about  responses  to  possible  adversaries? 


Discretionary  Decisions 


1 .  Snipers  in  area 


2.  Individual  hostiles  in  area 


3.  Hostile  crowds 


4.  Unknowns  in  area 


5.  Friendlies/Allied  troops  in  area 


6.  Villagers/bystanders  in  area 


7.  Hostages  in  area 


8.  Threatening  weapon  presentation 


9.  Non-threatening  weapon 
presentation 


10.  Fleeing  enemy 


11.  Wounded  enemy 


12.  Enemy  prisoners  of  war 


13.  Enemies  in  Allied  uniforms 


14.  Other  (specify) 


15,  Overall  TTES  capability: 
Discretionary  Decisions 


Potential  Usefulness 
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TTES  use:  mission  preview.  Mission  preview  is  a  mission  planning  function  for  a  specific  area 
and  military  operation.  No  dynamic  interactions  with  the  environment  or  potential  adversaries  are 
needed.  It  is  simply  a  virtual  walk-through  of  the  objective  area.  What  capabilities  should  TTES 
provide  for  mission  preview?  _ 


TI  ES  Mission  Preview  f  Current  Capability  |  Potential  Usefulness 


1.  Familiarize  and  retain 

familiarization _ 

2.  Visualize  approach  and  routes 


3.  Visualize  obstacles  and  danger 
zones 


4.  Visualize  potential  kill  zones 


5.  Evaluate  sectors  of  fire  for 
defense/offense 


6.  Locate  supplementary/altemate 
positions  for  defense/offense 


7.  Evaluate  courses  of  action 


8.  Solidify  plan  of  action 


9.  Overall  TTES  capability: 
Mission  Preview 


5.  TTES  use:  mission  rehearsal.  Mission  rehearsal  is  a  mission  preparation  function  — 

conducting  a  simulated  operation  or  tactical  sequence  in  a  specific  dynamic  environment.  Hostile 
and  neutral  forces  are  tailored  to  the  expected  adversary  profile.  The  goals  are  to  validate  a 
concept  of  operations  and  to  assess  the  scenario  and  potential  outcome.  What  capabilities  should 
TTES  provide  for  mission  rehearsal? 


TTES  Mission  Rehearsal 


1.  Immediate  action  reaction  to 
sniper  fire 


2.  Immediate  action  reaction  to 
ambush 


3.  Cross  danger  point 


4.  Deal  with  threat  point 


5.  React  to  hostile  presence 


6.  React  to  neutral  presence 


7.  Move  to  supplementary/altemate 
positions 


8.  Evaluate  movement  techniques 


9.  Compare  simulation  outcomes  to 
expectation 


10.  Evaluate  courses  of  action 


1 1 .  Evaluate  plan  of  action 


12.  Solidify  plan  of  action 


13.  Overall  TTES  capability: 
Mission  Rehearsal 
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B.  TTES  Objects,  Situations,  and  Events 

1.  Environments  to  simulate  in  TTES.  What  natural  and  man-made  environments  should  be 


simulated  in  the  TTES  synthetic  environment? 


TTES  Environ ments 

Potential  Useful  ness 

1.  Urban:  city 

2.  Urban:  village 

3.  Farmland 

4.  Desert 

5.  Jungle 

6.  Arctic/winter 

7.  NBC  environments 

8.  Federal  buildings 

9.  Prisons 

10.  Ships,  submarines 

11.  Other  (Specify) 

12.  Overall  TTES  capability: 
Environments 

l.a.  Realism  level  required  for  satisfactory  training.  How  good  is  “good  enough”?  Choose  one: 

1 .  Exact  replication  of  real-world  areas  and  environments. 

2.  Good  replication,  hard  to  tell  it  is  not  real. 

3.  Reasonable  replication,  appearance  about  as  expected  for  this  kind  of  area  or  environment 

4.  So-so  replication,  appearance  somewhat  realistic. 

5.  Crude  approximation,  just  enough  to  recognize  area  or  environment. 
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3.  Pistol  projectile  impacts 


4.  Mortar  impacts 


5.  Shoulder-launched  rocket  impacts 


6.  Grenade  impacts 


7.  Structure  fires 


8.  Reduced  visibility  (fog,  rain) 


9.  Other  (Specify) 


10.  Overall  TTES  capability: 
Environment  Effects 


2.a.  Realism  level  required  for  satisfactory  training.  How  good  is  “good  enough”?  Choose  one: 

1 .  Exact  replication  of  real-world  weapons  and  environmental  effects. 

2.  Good  replication,  hard  to  tell  it  is  not  real. 

3.  Reasonable  replication,  appearance  about  as  expected  for  this  weapon’s  or  environment’s  effects. 

4.  So-so  replication,  weapon  and  environmental  effects  somewhat  realistic. 

5.  Crude  approximation,  just  enough  to  know  where  weapon  has  struck  or  the  kind  of  environment. 


9.  Traps,  trip  wires,  mines 


10.  Explosions 


11.  Burning  objects 


12.  Incoming  fire 


13.  Target  hits 


14.  Debris 


15.  Rubble 


16.  Buildings 


17.  Doors  &  windows 


18.  Furniture 


19.  Other  (specify) 


20.  Overall  TTES  capability: 
General  Objects 


3.a.  Realism  level  required  for  satisfactory  training.  How  good  is  “good  enough”?  Choose  one: 

1 .  Exact  replication  of  real-world  objects. 

2.  Good  replication,  hard  to  tell  it  is  not  real. 

3.  Reasonable  replication,  appearance  about  as  expected  for  this  kind  of  object. 

4.  So-so  replication,  appearance  somewhat  realistic. 


5.  Crude  approximation,  just  enough  to  recognize  this  kind  of  object. 


4.  Simulated  humans  in  TTES  environment.  What  humans  should  be  present  in  the  TTES 
synthetic  environment  for  trainees  to  interact  with,  as  they  go  through  training  situations? 


TTES  Simulated  Humans 

Current  Capability 

Potential  Usefulness 

1.  Snipers 

2.  Hostiles 

3.  Unknowns 

4.  Friendlies 

' 

5.  Villagers/innocent  bystanders 

6.  Hostages 

7.  Other  (specify) 

8.  Overall  TTES  capability: 
Simulated  Humans 

4.a.  Realism  level  required  for  satisfactory  training.  How  good  is  “good  enough”?  Choose  one: 

1 .  Exact  replication  of  real-world  humans. 

2.  Good  replication,  hard  to  tell  it  is  not  real. 

3.  Reasonable  replication,  appearance  about  as  expected  for  this  kind  of  human. 

4.  So-so  replication,  appearance  somewhat  realistic. 

5.  Crude  approximation,  just  enough  to  recognize  kind  of  human. 
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5.  Weapons  in  TTES  environment.  What  weapon  systems  should  be  present  in  the  TTES  synthetic 


environment  for  trainees  to  practice  using  and  to  interact  with? 


i  TIES  Weapons 

Current  Capability 

Potential  Usefulness 

1.  Assault  rifles 

2.  Pistols 

3.  Submachine  guns 

4.  Machine  guns 

5.  Explosive  charges 

6.  Mortars 

7.  Flashbangs 

8.  Grenades 

9.  Satchel  charges 

10.  Squad  automatic  weapons 

11.  Shoulder-launched  missiles/ 
rockets 

12.  Knives 

13.  Other  (specify) 

14.  Overall  TTES  capability: 
Weapons 

S.a.  Realism  level  required  for  satisfactory  training.  How  good  is  “good  enough”?  Choose  one: 

1 .  Exact  replication  of  real  weapon. 

2.  Good  replication,  hard  to  tell  it  is  not  real. 

3.  Reasonable  replication,  appearance  about  as  expected  for  this  kind  of  weapon. 

4.  So-so  replication,  appearance  somewhat  realistic. 

5.  Crude  approximation,  just  enough  to  recognize  this  kind  of  weapon. 
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6.  Sounds  in  TTES  environment.  What  audible  sounds  should  be  present  in  the  TTES  synthetic 
environment  to  add  realism  to  training  situations? 


TTES  Sounds 


1 .  Explosions 


2.  Weapon  functioning 


3.  Trees  &  branches  snapping 


4.  Footsteps 


5.  Verbal  commands 


6.  Normal  conversations 


7.  Excited  chatter 


8.  Other  (specify) 


9.  Overall  TTES  capability: 
Simulated  Sounds 


6.a.  Realism  level  required  for  satisfactory  training.  How  good  is  “good  enough”?  Choose  one: 

1 .  Exact  replication  of  real-world  sounds. 

2.  Good  replication,  hard  to  tell  it  is  not  real. 

3 .  Reasonable  replication,  appearance  about  as  expected  for  this  kind  of  sound. 

4.  So-so  replication,  appearance  somewhat  realistic. 

5 .  Crude  approximation,  just  enough  to  recognize  this  kind  of  sound. 


C.  Trainee  and  Adversary  Movements  and  Positions 

l.a.  Trainee  coordinated  tactical  movements.  What  coordinated  tactical  movements  should  TTES 
allow  trainees  to  perform  in  the  simulator,  as  they  go  through  simulated  training  situations? 

Lb.  Simulated  adversary  coordinated  tactical  movements.  What  coordinated  tactical  movements 


should  TTES  simulate  for  hostiles,  neutrals,  and  friendlies,  for  various  training  situations? 


BBS  E  I 

Trainee  Pi;  ■  1 

I  Adversary::  ••  | 

Coordinated  Tactical  M 

0\ 

eme 

nts 

Current 

Capability 

Potential 

Usefulness 

Current 

Capability 

Potential 

Usefulness 

1.  Use  hand  &  arm  signals 

2.  Enter  building 

3.  Enter  room 

4.  Enter  ambush  site 

5.  Enter  vehicle 

6.  Patrol 

7.  Control  crowds/marshal  people 

8.  Rush  (fire  &  move) 

9.  Fire  &  maneuver 

10.  Cover  moving  buddies 

1 1 .  Pick  up/carry  injured  buddy 

12.  Other  (specify) 

13.  Overall  TTES  capability: 
Coordinated  Tactical  Movements 
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2. a.  Individual  trainee  body  movements.  What  tactically-correct  whole-body  movements  should 
TTES  allow  trainees  to  perform  in  the  simulator? 


2,b.  Simulated  adversary  movements.  What  whole-body  movements  should  TTES  simulate  for 
hostiles,  neutrals,  and  friendlies,  as  trainees  encounter  them  during  training  situations? _ 


Individual  Body  Movements 

1.  Squat/duck  walk 

2.  Kneel  down 

3.  Lie  down  on  front 

4.  Lie  down  on  back 

5.  Roll  right  &  left 

6.  Hit  &  roll 

7.  Stand  up 

8.  Lean  over 

9.  Rotate  body 

10.  Duck 

11.  Crawl 

12.  Walk  forwards 

13.  Walk  backwards 

14.  Run 

15.  Jump 

16.  Climb/crawl  over  obstacles 

17.  Dive  behind  cover 

18.  Other  (specify) 

19.  Overall  TTES  capability: 
Individual  Body  Movements 


3. a.  Trainee  firing  positions.  What  tactically-correct  firing  positions  and  other  body  positions  should 
TTES  allow  trainees  to  use  in  the  simulator? 


3.b.  Simulated  adversary  positions.  What  firing  positions  and  other  body  positions  should  TTES  be 
able  to  simulate  for  hostiles,  neutrals,  and  friendlies,  as  trainees  decide  whether  to  fire? 


4. a.  Trainee  arm  and  hand  movements.  What  tactically-correct  arm  and  hand  movements  should 
TTES  allow  trainees  to  perform  in  the  simulator,  as  they  move  through  training  situations? 


4  b.  Simulated  adversary  arm  and  hand  movements.  What  tactically-correct  arm  and  hand 
movements  should  TTES  be  able  to  simulate  for  hostiles,  neutrals,  and  friendlies,  as  trainees  go 
through  training  situations?  _ _ _ 


Trainee 

Adversary  | 

Arm  &  Hand  Movements 

Current 

Capability 

Potential 

Usefulness 

Current 

Capability 

Potential 

Usefulness 

1 .  Up  &  down 

2.  Left&  right 

3.  Diagonal  (up  &  to  the  right,  etc.) 

4.  Bend  elbows  &  wrists 

5.  Reach  for  objects 

6.  Grab  or  grasp  objects 

7.  Pick  up  objisets 

8.  Throw  objects  (e.g.,  grenades) 

9.  Change  hands 

10.  Push  objects  away 

11.  Open  doors 

12.  Open  windows 

13.  Draw  weapons 

14.  Manipulate  weapons 

15.  Shoot  weapons 

16.  Transition  between  weapons 

17.  Use  rifle  as  pugil 

18.  Other  (specify) 

19.  Overall  TTES  capability: 

Arm  &  Hand  Movements 

5. a.  Trainee  head  movements.  What  tactically-correct  trainee  head  movements  are  trainees  likely  to 
perform  and  for  which  TTES  should  provide  a  simulated  field  of  regard? 

5.b.  Simulated  adversary  head  movements.  What  tactically-correct  head  movements  should  TTES 
be  able  to  simulate  for  hostiles,  neutrals,  and  friendlies? _  _ 

:  :  :  '  :  - ;  ?  1 | :  i : !  |  j  i j j  \  \  ■  j  | !  j  j || ; ; ; .  |  i  j  j : i  | j  I  ii  1 !■ : :  il  Trainee  I :  gj I  ^  -j:  £ :  ^ 

Current  Potential  II  Current  Potential 

Head  Movements  Capability  |  Usefulness  ||  Capability  Usefulness 

1 .  Up  &  down 

2.  Left  &  right 

3.  Diagonal  (up  &  to  the  right,  etc.) 

4.  Slow  scan  of  area 

5.  Fast  tracking  of  moving  object 

6.  Look  over  shoulder/behind 

7.  Peek/look  around  comer 

8.  Peek/look  over  obstacle 

9.  Bend  &  look 

10.  Duck 

11.  Put  face  in  dirt 

12.  Aim  through  sight 

13.  Other  (specify) 

14.  Overall  TTES  capability: 

Head  Movements  _ 


6.  a.  Trainee  eye  movements.  What  tactically-correct  eye  movements  are  trainees  likely  to  perform 
and  for  which  TTES  should  provide  a  simulated  field  of  regard? 

6.b.  Simulated  adversary  eye  movements.  What  tactically-correct  eye  movements  should  TTES  be 
able  to  simulate  for  hostiles,  neutrals,  and  friendlies? 


_ EyeMaveaieatf 

1.  Up  &  down 

2.  Left  &  right 

3.  Diagonal  (up  &  to  the  right,  etc.) 

4.  Scan  area,  limited  head  motion 

5.  Fast  tracking  of  moving  objects 

6.  Through  peep  holes 

7.  Sight  alignment 

8.  Other  (specify) 

9.  Overall  TTES  capability:  Eye 
Movements 


j  Trainee 

pi; Carrettifc u  Potential:  j 

Capability  r-  U&fiitaeisl 


Adversar 


Potential 
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Appendix  B:  MISSION  PREVIEW  AND  MISSION  REHEARSAL 


The  Mission  Preview  and  Mission  Rehearsal  categories  were  not  included  in  the  1994 
evaluation  of  perceived  TTES  training  value.  As  the  year  progressed,  the  potential  usefulness  of 
these  possible  TTES  functions  became  obvious,  so  the  two  categories  were  added  to  the  1995 
questionnaire  for  evaluation  by  the  28  subject  matter  experts  who  participated  in  the  survey. 

In  addition,  a  second  group  completed  a  separate  questionnaire  dealing  only  with  Mission 
Preview  and  Mission  Rehearsal.  Twelve  enlisted  men  and  one  officer  from  the  Second  Marine 
Division,  Camp  Lejeune,  North  Carolina,  completed  this  questionnaire.  Five  of  these  had  seen 
combat  in  Haiti  or  Liberia;  the  rest  had  never  been  in  combat. 

These  survey  respondents  reviewed  a  map  of  the  objective  area,  which  was  the  Marine  Corps 
MOUT II  Quantico  Combat  Training  Village  modeled  in  TTES,  and  also  used  TTES  to  preview 
and  rehearse  a  mission.  That  mission  then  was  carried  out  as  an  exercise  in  the  Combat  Training 
Village,  prior  to  completion  of  the  questionnaire. 

Since  the  Mission  Preview  and  Mission  Rehearsal  capabilities  listed  on  the  questionnaire  for 
this  group  were  not  the  same  as  those  on  the  1995  training  value  survey  form,  results  could  not  be 
combined.  Instead,  calculated  ratings  are  provided  in  the  following  tables  for  that  separate  group 
of  respondents.  Responses  are  sorted  so  the  Total  Group’s  highest  rated  capabilities  are  at  the  top. 


1.  Using  TTES  for  Mission  Preview. 

Mission  preview  is  a  mission  planning  function  for  a  specific  area  and  military  operation.  No 
dynamic  interactions  with  the  environment  or  potential  adversaries  are  needed.  It  is  simply  a 
virtual  walk-through  of  the  objective  area.  What  capabilities  should  TTES  provide  for 
mission  preview? 


Potential  Usefulness  of  Listed  Factors  for  Mission  Preview 


Potential  Usefulness  Rating 

Mission  Preview 

Total  Group 

No  Combat 

Combat 

Familiarize  &  retain  familiarization  of  objective  area 

96.9 

100.0 

92.0 

96.9 

97.5 

96.0 

Evaluate  sectors  of  fire  for  offense/defense 

95.4 

100.0 

88.0 

Visualize  own  avenue  of  approach  and  routes 

95.4 

97.5 

92.0 

93.8 

95.0 

92.0 

Visualize  obstacles  to  movement  and  danger  zones 

90.8 

100.0 

76.0 

Evaluate  weapon  positions  for  defense/offense 

90.8 

87.5 

96.0 

Evaluate  courses  of  action 

87.7 

87.5 

88.0 

Visualize  check/reference  points 

86.2 

92.5 

76.0 

Locate  supplementary/a Itemate  positions  for 
defense/offense 

86.2 

92.5 

76.0 

Visualize  potential  kill  zones 

86.2 

87.5 

84.0 

Visualize  covered  routes 

86.2 

87.5 

84.0 

Visualize  obstacles  to  observation 

84.6 

85.0 

84.0 

Visualize  deadspace  for  weapons 

84.6 

80.0 

92.0 

Visualize  avenues  of  approach  against  you 

83.1 

85.0 

80.0 

Familiarize  with  floorplans  of  buildings  encountered 

83.1 

82.5 

84.0 

Potential  Overall  TTES  usefulness:  Mission  Preview 

100.0 

100.0 

100.0 
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Current  Capability  of  TIES  to  Emulate  Listed  Factors  for  Mission  Preview 


Current  Capability  Rating 

Mission  Preview 

Total  Group 

No  Combat 

Combat 

Visualize  own  avenue  of  approach  and  routes 

95.4 

95.0 

96.0 

Evaluate  sectors  of  fire  for  offense/defense 

83.1 

82.5 

84.0 

Solidify  plan  of  action 

83.1 

80.0 

88.0  I 

Familiarize  &  retain  familiarization  of  objective  area 

81.5 

80.0 

84.0 

Familiarize  with  building  layout  and  arrangement 

80.0 

77.5 

84.0 

Visualize  obstacles  to  movement  and  danger  zones 

78.5 

80.0 

76.0 

Locate  supplementary/altemate  positions  for 
defense/offense 

78.5 

80.0 

76.0 

Evaluate  courses  of  action 

75.4 

72.5 

80.0 

Visualize  potential  kill  zones 

73.8 

80.0 

64.0 

Visualize  check/reference  points 

73.8 

80.0 

64.0 

Evaluate  weapon  positions  for  defense/offense 

73.8 

70.0 

80.0 

Familiarize  with  floorplans  of  buildinqs  encountered 

70.8 

65.0 

80.0 

Visualize  covered  routes 

69.2 

75.0 

60.0 

Visualize  obstacles  to  observation 

69.2 

70.0 

68.0 

Visualize  avenues  of  approach  against  you 

64.6 

62.5 

68.0 

Visualize  deadspace  for  weapons 

60.0 

50.0 

76.0 

Current  Overall  TTES  capability:  Mission  Preview 

83.3 

80.0 

f  88.0 

1.  Using  TTES  for  Mission  Rehearsal. 

Mission  rehearsal  is  a  mission  preparation  function  —  conducting  a  simulated  operation  or 
tactical  sequence  in  a  specific  dynamic  environment.  Hostile  and  neutral  forces  are  tailored  to 
the  expected  adversary  profile.  The  goals  are  to  validate  a  concept  of  operations  and  to  assess 
the  scenario  and  potential  outcome.  What  capabilities  should  TTES  provide  for  mission 
rehearsal? 


Potential  Usefulness  of  Listed  Factors  for  Mission  Rehearsal 


Potential  Usefulness  Hi 

iting 

Mission  Rehearsal 

Total  Group 

No  Combat 

Combat 

React  to  hostile  presence 

98.3 

100.0 

96.0 

React  to  neutral  presence 

96.7 

100.0 

92.0 

Try  out  avenue  of  approach 

95.0 

97.1 

92.0  ! 

Cross  danqer  point 

93.3 

100.0 

84.0 

Evaluate  pian  of  action 

93.3 

97.1 

88.0 

Try  out  defensive/offensive  actions 

91.7 

97.1 

84.0 

Deal  with  threat  point 

90.0 

100.0 

76.0 

Evaluate  courses  of  action 

90.0 

88.6 

92.0 

Move  to  supplementary/altemate  positions 

88.3 

88.6 

88.0 

Compare  simulation  outcomes  to  expectation 

85.5 

85.7 

85.0 

Evaluate  movement  techniques 

81.7 

88.6 

72.0 

Immediate  action  reaction  to  ambush 

81.7 

82.9 

80.0 

i.-i.'iMiji.iki.i.iim— 

81.7 

74.3 

92.0 

React  to  beinq  probed 

80.0 

88.6 

68.0 

Immediate  action  reaction  to  sniper  fire 

75.0 

82.9 

64.0 

Potential  Overall  TTES  usefulness: 

Mission  Rehearsal 

98.0 

100.0 

96.0 

Current  Capability  of  TTES  to  Emulate  Listed  Factors  for  Mission  Rehearsal 


Current  Capability  Rating 

Mission  Rehearsal 

Total  Group 

No  Combat 

Combat 

React  to  neutral  presence 

86.7 

85.7 

88.0 

Try  out  avenue  of  approach 

83.3 

82.9 

84.0 

83.3 

82.9 

84.0 

Evaluate  plan  of  action 

83.3 

82.9 

84.0 

Evaluate  courses  of  action 

81.7 

82.9 

80.0 

React  to  hostile  presence 

81.7 

77.1 

88.0 

Deal  with  threat  point 

78.3 

80.0 

76.0 

Try  out  defensive/offensive  actions 

76.7 

77.1 

76.0 

75.0 

68.6 

84.0 

Compare  simulation  outcomes  to  expectation 

72.7 

68.6 

80.0 

Evaluate  movement  techniques 

71.7 

77.1 

64.0  ! 

Move  to  supplementary/altemate  positions 

71.7 

65.7 

80.0 

Immediate  action  reaction  to  ambush 

70.0 

74.3 

64.0 

React  to  being  probed 

66.7 

68.6 

64.0 

Immediate  action  reaction  to  sniper  fire 

61.7 

65.7 

56.0 

Current  Overall  TTES  capability: 

Mission  Rehearsal 

82.0 

80.0 

84.0 

The  respondents  for  this  survey  also  were  asked  to  provide  comments  on  whether  the  use  of 
TTES  improved  their  ability  to  perform  or  execute  the  mission.  They  were  asked  to  compare  the 
virtual  preparation  with  the  live  exercise  and  to  report  whether  they  were  better  prepared  and  more 
confident,  and  whether  they  knew  what  they  were  doing  as  a  result  of  the  opportunity  to  preview 
and  rehearse  the  mission.  Responses  included  the  following. 

•  A  combination  of  both  the  map  and  technology  helps  [make]  the  mission  a  little  easier  to 
understand. 

•  It  gives  a  step  by  step  of  what  would  be  encountered.  I  felt  I  knew  the  terrain.  It  was  clear 
what  the  best  way  of  approach  was. 

•  I  had  an  idea  of  where  snipers  could  position  themselves  and  I  knew  exactly  how  to  provide 
security  for  my  team. 

•  It  gave  us  a  visual  picture  of  the  whole  town.  I  think  that  a  visual  picture  and  a  walk  through 
the  town  through  the  system  is  lOOx  better  than  doing  a  map  recon  while  you  [are]  trying  to 
explain  everything  to  your  Sqd  [Squad]  or  Fr  Tm  [Fire  Team],  You  can  actually  show  them  so 
they  know.  And  there  is  no  doubt  in  your  mind  that  they  [may  not]  know  what  they  are  doing. 

I  knew  exactly  what  I  was  doing  and  what  my  tasks  were. 

•  My  team  leader  told  me  and  he  backed  it  up  with  the  computer. 

•  It  prepared  me  for  the  mission:  where  to  go,  etc.  It  gave  me  a  heads  up  on  what  I  was  going 
against  before  it  actually  happened. 

•  The  TTES  helped  a  lot  to  prepare  me/Squad  in  knowing  where  each  individual  needed  to  be 
before  the  mission  started.  I  felt  confident  in  knowing  building  structure/floor  plans,  but  the 
buildings  need  external  identifiers  (i.e..  Hotel,  Bank,  etc  ). 
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•  I  recalled  all  of  the  info,  because  I  [had]  seen  it.  It  improved  my  ability  to  perform  by  having  a 
mental  picture. 

•  It  was  like  replaying  a  movie  in  my  mind.  I  knew  exactly  what  I  was  supposed  to  do  and  also 
where  all  of  my  team  and  I  were  supposed  to  be....  My  team  knew  also  because  we  saw  it  on 
the  screen.  We  all  saw  what  we  had  to  do  exactly  and  I  knew  my  team  was  LOCKED  ON! ! 

•  The  only  problem  was  that  one  of  the  buildings  you  see  on  the  screen  is  not  the  same  as  in 
reality.  It  did  provide  me  with  a  better  mental  picture  that  I  was  able  to  apply  once  we 
encountered  the  building.  The  experience  improved  my  ability  to  perform  and  execute  the 
mission  for  the  simple  reason  that  I  was  able  to  have  a  detailed  mental  picture  of  the  area 
before  I  even  [had]  seen  it  in  reality.  I  did  know  what  I  was  doing  because  I  went  over  it  with 
my  team  and  SQ  on  the  virtual  screen  a  few  times  before  doing  it. 

•  Great  tool  for  evaluating  team  performance.  Controller  screen  gives  a  good  view. 
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